vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #424
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.QnwTfvYBUE', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: a14ca34f-8c5a-480e-931e-7cf7c398cd6016(finished: start all VMs, in 48.61 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fpli9vynwlp4k9slgzrmqljc5j0jlrvd-nixos-system-gitlab-test/init regInfo=/nix/store/8ngs82cz8hynxrla3rl9p2wfzclcfzmh-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000001] kvm-clock: using sched offset of 492831977 cycles65gitlab # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000007] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000944] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000981] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000984] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.001036] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.003146] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.003161] Using GB pages for direct mapping73gitlab # [ 0.003385] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.003394] ACPI: Early table checksum verification disabled75gitlab # [ 0.003398] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.003402] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.003407] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.003417] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.003420] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.003423] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.003425] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.003427] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.003430] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.003431] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.003432] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.003432] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.003433] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.003433] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003734] No NUMA configuration found90gitlab # [ 0.003735] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003738] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.006719] Zone ranges:93gitlab # [ 0.006720] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.006722] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.006724] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.006725] Device empty97gitlab # [ 0.006726] Movable zone start for each node98gitlab # [ 0.006727] Early memory node ranges99gitlab # [ 0.006727] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.006728] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.006729] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.006731] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.006760] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.007109] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.127105] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.127568] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.127584] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.127621] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.127624] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.127626] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.127628] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.127629] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.127629] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.127632] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.127633] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.127640] TSC deadline timer available117gitlab # [ 0.127645] CPU topo: Max. logical packages: 1118gitlab # [ 0.127646] CPU topo: Max. logical dies: 1119gitlab # [ 0.127646] CPU topo: Max. dies per package: 1120gitlab # [ 0.127650] CPU topo: Max. threads per core: 1121gitlab # [ 0.127650] CPU topo: Num. cores per package: 4122gitlab # [ 0.127651] CPU topo: Num. threads per package: 4123gitlab # [ 0.127651] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.127678] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.127696] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.127698] kvm-guest: setup PV sched yield127gitlab # [ 0.127714] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.127716] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.127717] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.127719] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.127721] Booting paravirtualized kernel on KVM132gitlab # [ 0.127725] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.132199] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.134570] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.134624] kvm-guest: PV spinlocks enabled136gitlab # [ 0.134626] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.134629] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fpli9vynwlp4k9slgzrmqljc5j0jlrvd-nixos-system-gitlab-test/init regInfo=/nix/store/8ngs82cz8hynxrla3rl9p2wfzclcfzmh-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.134729] Unknown kernel command line parameters "regInfo=/nix/store/8ngs82cz8hynxrla3rl9p2wfzclcfzmh-closure-info/registration", will be passed to user space.139gitlab # [ 0.134743] random: crng init done140gitlab # [ 0.134744] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.144152] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.149126] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.149207] software IO TLB: area num 4.144gitlab # [ 0.229690] Fallback order for Node 0: 0145gitlab # [ 0.229699] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.229701] Policy zone: Normal147gitlab # [ 0.232339] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.243073] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.257947] allocated 12582912 bytes of page_ext150gitlab # [ 0.268215] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.268217] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.269144] Dynamic Preempt: lazy153gitlab # [ 0.269333] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.269334] rcu: RCU event tracing is enabled.155gitlab # [ 0.269335] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.269337] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.269337] Rude variant of Tasks RCU enabled.158gitlab # [ 0.269337] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.269338] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.269339] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.269362] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.269364] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.269365] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.273811] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.274128] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.274138] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.274263] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.277936] Console: colour VGA+ 80x25169gitlab # [ 0.277939] printk: legacy console [tty0] enabled170gitlab # [ 0.308925] printk: legacy console [ttyS0] enabled171gitlab # [ 0.417280] ACPI: Core revision 20250807172gitlab # [ 0.418260] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.419877] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.420897] x2apic enabled175gitlab # [ 0.421662] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.422610] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.423894] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.425391] kvm-guest: setup PV IPIs179gitlab # [ 0.426999] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.428050] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.429792] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.431880] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.432928] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.434789] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.435794] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.436790] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.437790] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.438789] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.439790] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.440790] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.442798] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.443569] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.444790] active return thunk: srso_alias_return_thunk194gitlab # [ 0.445812] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.446790] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.447790] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.449614] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.450627] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.451618] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.452790] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.453790] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.454790] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.455790] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.456789] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.457789] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.458790] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.489654] Freeing SMP alternatives memory: 44K208gitlab # [ 0.490423] pid_max: default: 32768 minimum: 301209gitlab # [ 0.490919] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.491898] landlock: Up and running.211gitlab # [ 0.493269] Yama: becoming mindful.212gitlab # [ 0.494002] LSM support for eBPF active213gitlab # [ 0.495035] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.495951] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.499197] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.500385] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.500799] ... version: 2218gitlab # [ 0.501568] ... bit width: 48219gitlab # [ 0.501792] ... generic counters: 6220gitlab # [ 0.502554] ... generic bitmap: 000000000000003f221gitlab # [ 0.502793] ... fixed-purpose counters: 0222gitlab # [ 0.503562] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.503792] ... value mask: 0000ffffffffffff224gitlab # [ 0.504730] ... max period: 00007fffffffffff225gitlab # [ 0.505533] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.505921] signal: max sigframe size: 3376227gitlab # [ 0.506731] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.507430] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.507976] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.513542] smp: Bringing up secondary CPUs ...231gitlab # [ 0.514204] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.514801] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.517891] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.519512] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.520295] Memory: 6011580K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.521793] devtmpfs: initialized237gitlab # [ 0.522831] x86/mm: Memory block size: 128MB238gitlab # [ 0.525918] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.526878] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.527906] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.529090] PM: RTC time: 18:56:15, date: 2026-10-04242gitlab # [ 0.533067] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.535205] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.535864] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.537960] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.538804] audit: initializing netlink subsys (disabled)247gitlab # [ 0.539979] audit: type=2000 audit(1791140175.596:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.540159] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.541793] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.542792] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.543792] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.544792] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.545817] cpuidle: using governor menu254gitlab # [ 0.548056] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.549137] PCI: Using configuration type 1 for base access256gitlab # [ 0.549794] PCI: Using configuration type 1 for extended access257gitlab # [ 0.551047] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.553863] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.554793] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.555792] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.556792] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.561865] ACPI: Added _OSI(Module Device)263gitlab # [ 0.563804] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.565798] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.572799] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.580377] ACPI: Interpreter enabled267gitlab # [ 0.581855] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.583800] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.585886] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.588800] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.592131] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.605081] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.607847] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.611675] acpiphp: Slot [3] registered275gitlab # [ 0.612825] acpiphp: Slot [4] registered276gitlab # [ 0.613880] acpiphp: Slot [5] registered277gitlab # [ 0.615904] acpiphp: Slot [6] registered278gitlab # [ 0.616876] acpiphp: Slot [7] registered279gitlab # [ 0.618874] acpiphp: Slot [8] registered280gitlab # [ 0.619886] acpiphp: Slot [9] registered281gitlab # [ 0.621866] acpiphp: Slot [10] registered282gitlab # [ 0.622874] acpiphp: Slot [11] registered283gitlab # [ 0.623881] acpiphp: Slot [12] registered284gitlab # [ 0.625889] acpiphp: Slot [13] registered285gitlab # [ 0.626839] acpiphp: Slot [14] registered286gitlab # [ 0.628835] acpiphp: Slot [15] registered287gitlab # [ 0.629669] acpiphp: Slot [16] registered288gitlab # [ 0.630349] acpiphp: Slot [17] registered289gitlab # [ 0.630813] acpiphp: Slot [18] registered290gitlab # [ 0.631554] acpiphp: Slot [19] registered291gitlab # [ 0.632813] acpiphp: Slot [20] registered292gitlab # [ 0.633544] acpiphp: Slot [21] registered293gitlab # [ 0.633813] acpiphp: Slot [22] registered294gitlab # [ 0.634560] acpiphp: Slot [23] registered295gitlab # [ 0.635821] acpiphp: Slot [24] registered296gitlab # [ 0.636553] acpiphp: Slot [25] registered297gitlab # [ 0.636812] acpiphp: Slot [26] registered298gitlab # [ 0.637571] acpiphp: Slot [27] registered299gitlab # [ 0.638820] acpiphp: Slot [28] registered300gitlab # [ 0.639559] acpiphp: Slot [29] registered301gitlab # [ 0.639813] acpiphp: Slot [30] registered302gitlab # [ 0.640557] acpiphp: Slot [31] registered303gitlab # [ 0.641808] PCI host bridge to bus 0000:00304gitlab # [ 0.642527] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.642793] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.643792] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.645792] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.646854] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.650825] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.653841] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.658369] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.662928] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.670869] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.672867] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.675798] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.678798] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.681798] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.684276] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.690877] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.695741] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.699358] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.701808] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.704236] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.722885] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.724808] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.725816] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.728797] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.734626] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.745816] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.747813] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.750853] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.775817] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.778480] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.782812] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.783730] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.784598] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.787729] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.794815] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.795757] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.796586] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.799703] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.805802] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.806709] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.807601] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.810027] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.816133] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.817500] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.818816] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.821130] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.824814] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.825723] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.827815] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.830045] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.833608] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.834507] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.834815] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.836799] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.840017] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.841807] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.842815] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.846032] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.849811] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.850780] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.851602] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.855072] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.857803] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.859800] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.860815] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.866620] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.867877] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.868946] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.870251] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.871879] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.876153] iommu: Default domain type: Translated375gitlab # [ 0.878804] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.883214] ACPI: bus type USB registered377gitlab # [ 0.885023] usbcore: registered new interface driver usbfs378gitlab # [ 0.887842] usbcore: registered new interface driver hub379gitlab # [ 0.889832] usbcore: registered new device driver usb380gitlab # [ 0.894765] NetLabel: Initializing381gitlab # [ 0.896797] NetLabel: domain hash size = 128382gitlab # [ 0.898802] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.902104] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.903845] PCI: Using ACPI for IRQ routing385gitlab # [ 0.907992] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.908789] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.908789] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.918818] vgaarb: loaded389gitlab # [ 0.920449] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.922799] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.931007] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.936043] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.937518] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.940963] pnp: PnP ACPI init395gitlab # [ 0.944067] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.958243] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.961991] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.964561] NET: Registered PF_INET protocol family399gitlab # [ 0.969846] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 1.002742] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 1.004331] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 1.005718] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 1.009578] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 1.010992] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 1.012522] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 1.014179] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 1.015680] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 1.017058] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 1.018078] NET: Registered PF_XDP protocol family410gitlab # [ 1.018963] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 1.020023] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 1.021085] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 1.022241] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 1.023447] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 1.024959] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 1.025974] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 1.028522] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 1.031215] PCI: CLS 0 bytes, default 64419gitlab # [ 1.032096] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 1.032620] Trying to unpack rootfs image as initramfs...421gitlab # [ 1.033203] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 1.033336] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 1.056817] Initialise system trusted keyrings424gitlab # [ 1.058081] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 1.069828] Key type asymmetric registered426gitlab # [ 1.070613] Asymmetric key parser 'x509' registered427gitlab # [ 1.071592] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 1.073274] io scheduler mq-deadline registered429gitlab # [ 1.074164] io scheduler kyber registered430gitlab # [ 1.079294] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 1.080955] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 1.083476] Linux agpgart interface v0.103433gitlab # [ 1.084364] ACPI: bus type drm_connector registered434gitlab # [ 1.087306] usbcore: registered new interface driver usbserial_generic435gitlab # [ 1.088522] usbserial: USB Serial support registered for generic436gitlab # [ 1.089614] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 1.089614] Please enable it if your BIOS has the CPPC option.438gitlab # [ 1.092104] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 1.093595] drop_monitor: Initializing network drop monitor service440gitlab # [ 1.094957] NET: Registered PF_INET6 protocol family441gitlab # [ 1.096562] Segment Routing with IPv6442gitlab # [ 1.097415] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.099166] IPI shorthand broadcast: enabled444gitlab # [ 1.103696] sched_clock: Marking stable (956014248, 147651602)->(1129497232, -25831382)445gitlab # [ 1.105648] registered taskstats version 1446gitlab # [ 1.106803] Loading compiled-in X.509 certificates447gitlab # [ 1.116842] Demotion targets for Node 0: null448gitlab # [ 1.118072] Key type .fscrypt registered449gitlab # [ 1.118852] Key type fscrypt-provisioning registered450gitlab # [ 1.119946] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.120967] ima: Allocated hash algorithm: sha1452gitlab # [ 1.121864] ima: No architecture policies found453gitlab # [ 1.123138] PM: Magic number: 14:126:948454gitlab # [ 1.123947] tty tty5: hash matches455gitlab # [ 1.124675] processor cpu3: hash matches456gitlab # [ 1.125501] memory memory1: hash matches457gitlab # [ 1.127218] RAS: Correctable Errors collector initialized.458gitlab # [ 1.131934] clk: Disabling unused clocks459gitlab # [ 1.132736] PM: genpd: Disabling unused power domains460gitlab # [ 1.350687] Freeing initrd memory: 29392K461gitlab # [ 1.354651] Freeing unused decrypted memory: 2028K462gitlab # [ 1.357813] Freeing unused kernel image (initmem) memory: 3640K463gitlab # [ 1.358997] Write protecting the kernel read-only data: 32768k464gitlab # [ 1.361216] Freeing unused kernel image (text/rodata gap) memory: 1220K465gitlab # [ 1.363040] Freeing unused kernel image (rodata/data gap) memory: 760K466gitlab # [ 1.425323] x86/mm: Checked W+X mappings: passed, no W+X pages found.467gitlab # [ 1.426494] Run /init as init process468gitlab # [ 1.441194] systemd[1]: Inserted module 'autofs4'469gitlab # [ 1.465100] fuse: init (API version 7.45)470gitlab # [ 1.473868] ACPI: \_SB_.LNKC: Enabled at IRQ 10471gitlab # [ 1.481571] ACPI: \_SB_.LNKA: Enabled at IRQ 10472gitlab # [ 1.485117] ACPI: \_SB_.LNKB: Enabled at IRQ 11473gitlab # [ 1.527540] systemd[1]: Successfully made /usr/ read-only.474gitlab # [ 1.863268] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)475gitlab # [ 1.903316] systemd[1]: Detected virtualization kvm.476gitlab # [ 1.905534] systemd[1]: Detected architecture x86-64.477gitlab # [ 1.907757] systemd[1]: Running in initrd.478gitlab # [ 1.910268] systemd[1]: Initializing machine ID from random generator.479gitlab # [ 1.913204] systemd[1]: Hostname set to <gitlab>.480gitlab # [ 2.024148] systemd[1]: bpf-restrict-fs: LSM BPF program attached481gitlab # [ 2.084053] systemd[1]: Queued start job for default target Initrd Default Target.482gitlab # [ 2.092259] systemd[1]: Created slice Slice /system/modprobe.483gitlab # [ 2.093499] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.484gitlab # [ 2.095584] systemd[1]: Expecting device /dev/disk/by-label/nix-store...485gitlab # [ 2.097933] systemd[1]: Expecting device /dev/disk/by-label/nixos...486gitlab # [ 2.100115] systemd[1]: Reached target Path Units.487gitlab # [ 2.101683] systemd[1]: Reached target Slice Units.488gitlab # [ 2.102602] systemd[1]: Reached target Swaps.489gitlab # [ 2.103464] systemd[1]: Reached target Timer Units.490gitlab # [ 2.104494] systemd[1]: Listening on D-Bus System Message Bus Socket.491gitlab # [ 2.105806] systemd[1]: Listening on Journal Socket (/dev/log).492gitlab # [ 2.106988] systemd[1]: Listening on Journal Sockets.493gitlab # [ 2.108000] systemd[1]: Listening on udev Control Socket.494gitlab # [ 2.109058] systemd[1]: Listening on udev Kernel Socket.495gitlab # [ 2.110038] systemd[1]: Reached target Socket Units.496gitlab # [ 2.112319] systemd[1]: Starting Create List of Static Device Nodes...497gitlab # [ 2.114394] systemd[1]: Starting Load Kernel Module 9pnet_virtio...498gitlab # [ 2.116453] systemd[1]: Starting Load Kernel Module configfs...499gitlab # [ 2.121760] systemd[1]: Starting Journal Service...500gitlab # [ 2.126229] systemd[1]: Starting Load Kernel Modules...501gitlab # [ 2.127329] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os502gitlab # [ 2.133845] systemd[1]: Starting Coldplug All udev Devices...503gitlab # [ 2.142118] systemd[1]: Finished Create List of Static Device Nodes.504gitlab # [ 2.157330] systemd-journald[90]: Collecting audit messages is disabled.505gitlab # [ 2.157871] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...506gitlab # [ 2.160228] systemd[1]: modprobe@configfs.service: Deactivated successfully.507gitlab # [ 2.162389] netfs: FS-Cache loaded508gitlab # [ 2.166087] systemd[1]: Finished Load Kernel Module configfs.509gitlab # [ 2.167633] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config510gitlab # [ 2.175833] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.511gitlab # [ 2.176424] 9pnet: Installing 9P2000 support512gitlab # [ 2.185802] systemd[1]: Starting Create Static Device Nodes in /dev...513gitlab # [ 2.187315] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.514gitlab # [ 2.193127] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.515gitlab # [ 2.194854] systemd[1]: Finished Load Kernel Module 9pnet_virtio.516gitlab # [ 2.199741] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev517gitlab # [ 2.201518] systemd[1]: Finished Create Static Device Nodes in /dev.518gitlab # [ 2.203448] systemd[1]: Finished Load Kernel Modules.519gitlab # [ 2.204779] systemd[1]: Started Journal Service.520gitlab # [ 2.057201] systemd-modules-load[92]: Using 4 probe threads521gitlab # [ 2.059275] systemd-modules-load[92]: Inserted module 'virtio_balloon'522gitlab # [ 2.060455] systemd-modules-load[92]: Inserted module 'virtio_gpu'523gitlab # [ 2.061509] systemd-modules-load[92]: Inserted module 'dm_mod'524gitlab # [ 2.062542] systemd[1]: Reached target Preparation for Local File Systems.525gitlab # [ 2.063746] systemd[1]: Reached target Local File Systems.526gitlab # [ 2.064804] systemd[1]: Starting Apply Kernel Variables...527gitlab # [ 2.065821] systemd[1]: Starting Create System Files and Directories...528gitlab # [ 2.075109] systemd[1]: Starting Rule-based Manager for Device Events and Files...529gitlab # [ 2.082262] systemd[1]: Finished Apply Kernel Variables.530gitlab # [ 2.083670] systemd[1]: Finished Create System Files and Directories.531gitlab # [ 2.100267] systemd-udevd[104]: Using default interface naming scheme 'v261'.532gitlab # [ 2.110511] systemd[1]: Started Rule-based Manager for Device Events and Files.533gitlab # [ 2.130217] systemd[1]: Finished Coldplug All udev Devices.534gitlab # [ 2.132198] systemd[1]: Reached target System Initialization.535gitlab # [ 2.133069] systemd[1]: Reached target Basic System.536gitlab # [ 2.358584] virtio_blk virtio4: 4/0/0 default/read/poll queues537gitlab # [ 2.369627] SCSI subsystem initialized538gitlab # [ 2.370991] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)539gitlab # [ 2.376443] uhci_hcd 0000:00:01.2: UHCI Host Controller540gitlab # [ 2.378163] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1541gitlab # [ 2.379570] uhci_hcd 0000:00:01.2: detected 2 ports542gitlab # [ 2.379600] virtio_blk virtio5: 4/0/0 default/read/poll queues543gitlab # [ 2.381242] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140544gitlab # [ 2.383290] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18545gitlab # [ 2.384741] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1546gitlab # [ 2.386112] usb usb1: Product: UHCI Host Controller547gitlab # [ 2.386949] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd548gitlab # [ 2.388228] usb usb1: SerialNumber: 0000:00:01.2549gitlab # [ 2.392106] virtio_blk virtio5: [vdb] 11952896 512-byte logical blocks (6.12 GB/5.70 GiB)550gitlab # [ 2.397431] hub 1-0:1.0: USB hub found551gitlab # [ 2.398332] hub 1-0:1.0: 2 ports detected552gitlab # [ 2.416642] scsi host0: ata_piix553gitlab # [ 2.417637] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12554gitlab # [ 2.418359] scsi host1: ata_piix555gitlab # [ 2.419706] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0556gitlab # [ 2.421102] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0557gitlab # [ 2.421109] serio: i8042 KBD port at 0x60,0x64 irq 1558gitlab # [ 2.423040] serio: i8042 AUX port at 0x60,0x64 irq 12559gitlab # [ 2.302373] (udev-worker)[118]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.560gitlab # [ 2.305116] (udev-worker)[122]: Network interface NamePolicy= disabled on kernel command line.561gitlab # [ 2.306352] (udev-worker)[118]: Network interface NamePolicy= disabled on kernel command line.562gitlab # [ 2.308333] systemd[1]: Starting Virtual Console Setup...563gitlab # [ 2.324763] systemd[1]: Found device /dev/disk/by-label/nixos.564gitlab # [ 2.326140] systemd[1]: Reached target Initrd Root Device.565gitlab # [ 2.327076] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...566gitlab # [ 2.330643] systemd[1]: Found device /dev/disk/by-label/nix-store.567gitlab # [ 2.333065] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones.568gitlab # [ 2.337656] systemd[1]: Finished Virtual Console Setup.569gitlab # [ 2.353490] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks570gitlab # [ 2.357390] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.571gitlab # [ 2.582818] ata2: found unknown device (class 0)572gitlab # [ 2.584346] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100573gitlab # [ 2.586816] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5574gitlab # [ 2.614459] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0575gitlab # [ 2.632019] usb 1-1: new full-speed USB device number 2 using uhci_hcd576gitlab # [ 2.646527] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray577gitlab # [ 2.671999] cdrom: Uniform CD-ROM driver Revision: 3.20578gitlab # [ 2.799433] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00579gitlab # [ 2.802502] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10580gitlab # [ 2.805177] usb 1-1: Product: QEMU USB Tablet581gitlab # [ 2.806853] usb 1-1: Manufacturer: QEMU582gitlab # [ 2.808298] usb 1-1: SerialNumber: 28754-0000:00:01.2-1583gitlab # [ 2.839573] hid: raw HID events driver (C) Jiri Kosina584gitlab # [ 2.852965] usbcore: registered new interface driver usbhid585gitlab # [ 2.855136] usbhid: USB HID core driver586gitlab # [ 2.864543] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2587gitlab # [ 2.868245] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0588gitlab # [ 2.991224] systemd[1]: Mounting /sysroot...589gitlab # [ 3.302886] EXT4-fs (vda): mounted filesystem a14ca34f-8c5a-480e-931e-7cf7c398cd60 r/w with ordered data mode. Quota mode: none.590gitlab # [ 3.162834] systemd[1]: Mounted /sysroot.591gitlab # [ 3.165180] systemd[1]: Reached target Initrd Root File System.592gitlab # [ 3.170759] systemd[1]: Mounting /sysroot/nix/.ro-store...593gitlab # [ 3.178884] systemd[1]: Mounting /sysroot/nix/.rw-store...594gitlab # [ 3.182639] systemd[1]: Mounting /sysroot/run...595gitlab # [ 3.187297] systemd[1]: Mounting /sysroot/tmp/shared...596gitlab # [ 3.341523] erofs (device vdb): mounted with root inode @ nid 128.597gitlab # [ 3.195472] systemd[1]: Mounting /sysroot/tmp/xchg...598gitlab # [ 3.202862] systemd[1]: Starting Mountpoints Configured in the Real Root...599gitlab # [ 3.207791] systemd[1]: Mounted /sysroot/nix/.ro-store.600gitlab # [ 3.209737] systemd[1]: Mounted /sysroot/nix/.rw-store.601gitlab # [ 3.359092] 9p: Installing v9fs 9p2000 file system support602gitlab # [ 3.212529] systemd[1]: Mounted /sysroot/run.603gitlab # [ 3.216973] systemd[1]: Starting rw-sysroot-nix-store.service...604gitlab # [ 3.220118] systemd[1]: Mounted /sysroot/tmp/xchg.605gitlab # [ 3.223692] systemd[1]: Mounted /sysroot/tmp/shared.606gitlab # [ 3.225528] systemd-sysroot-fstab-check[177]: /sysroot should be mounted in the initrd, will request daemon-reload.607gitlab # [ 3.228150] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.608gitlab # [ 3.230022] systemd[1]: Finished rw-sysroot-nix-store.service.609gitlab # [ 3.231493] systemd[1]: Reload requested from client PID 177 ('systemd-sysroot') (unit initrd-parse-etc.service)...610gitlab # [ 3.233169] systemd[1]: Reloading...611gitlab # [ 3.293239] systemd[1]: Reloading finished in 65 ms.612gitlab # [ 3.313472] systemd-sysroot-fstab-check[177]: Requesting initrd-fs.target/start/replace...613gitlab # [ 3.316607] systemd-sysroot-fstab-check[177]: Requesting swap.target/start/replace...614gitlab # [ 3.319509] systemd[1]: Starting rw-sysroot-nix-store.service...615gitlab # [ 3.321770] systemd[1]: initrd-parse-etc.service: Deactivated successfully.616gitlab # [ 3.324316] systemd[1]: Finished Mountpoints Configured in the Real Root.617gitlab # [ 3.326667] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.618gitlab # [ 3.349310] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.619gitlab # [ 3.351575] systemd[1]: Finished rw-sysroot-nix-store.service.620gitlab # [ 3.991695] systemd[1]: Mounting /sysroot/nix/store...621gitlab # [ 4.047120] systemd[1]: Mounted /sysroot/nix/store.622gitlab # [ 4.049173] systemd[1]: Reached target Initrd File Systems.623gitlab # [ 4.051252] systemd[1]: Starting Find NixOS closure...624gitlab # [ 4.053271] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...625gitlab # [ 4.077294] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.626gitlab # [ 4.080137] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.627gitlab # [ 4.099162] systemd[1]: Finished Find NixOS closure.628gitlab # [ 4.100967] systemd[1]: Reached target Initrd Default Target.629gitlab # [ 4.102991] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...630gitlab # [ 4.129790] systemd[1]: Stopped target Initrd Default Target.631gitlab # [ 4.131834] systemd[1]: Stopped target Basic System.632gitlab # [ 4.133577] systemd[1]: Stopped target Initrd Root Device.633gitlab # [ 4.135459] systemd[1]: Stopped target Path Units.634gitlab # [ 4.137199] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.635gitlab # [ 4.139650] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.636gitlab # [ 4.142250] systemd[1]: Stopped target Slice Units.637gitlab # [ 4.144194] systemd[1]: Stopped target Socket Units.638gitlab # [ 4.146186] systemd[1]: Stopped target System Initialization.639gitlab # [ 4.148311] systemd[1]: Stopped target Swaps.640gitlab # [ 4.149991] systemd[1]: Stopped target Timer Units.641gitlab # [ 4.151927] systemd[1]: dbus.socket: Deactivated successfully.642gitlab # [ 4.154139] systemd[1]: Closed D-Bus System Message Bus Socket.643gitlab # [ 4.156286] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.644gitlab # [ 4.159181] systemd[1]: Stopped Find NixOS closure.645gitlab # [ 4.161447] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio646gitlab # [ 4.165593] systemd[1]: Starting rw-sysroot-nix-store.service...647gitlab # [ 4.167617] systemd[1]: systemd-sysctl.service: Deactivated successfully.648gitlab # [ 4.169462] systemd[1]: Stopped Apply Kernel Variables.649gitlab # [ 4.170955] systemd[1]: systemd-modules-load.service: Deactivated successfully.650gitlab # [ 4.172802] systemd[1]: Stopped Load Kernel Modules.651gitlab # [ 4.174110] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.652gitlab # [ 4.176080] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.653gitlab # [ 4.177993] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.654gitlab # [ 4.179806] systemd[1]: Stopped Create System Files and Directories.655gitlab # [ 4.181325] systemd[1]: Stopped target Local File Systems.656gitlab # [ 4.182669] systemd[1]: Stopped target Preparation for Local File Systems.657gitlab # [ 4.184323] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.658gitlab # [ 4.186126] systemd[1]: Stopped Coldplug All udev Devices.659gitlab # [ 4.187240] systemd[1]: Stopping Rule-based Manager for Device Events and Files...660gitlab # [ 4.188359] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.661gitlab # [ 4.189416] systemd[1]: Stopped Virtual Console Setup.662gitlab # [ 4.190200] systemd[1]: initrd-cleanup.service: Deactivated successfully.663gitlab # [ 4.191219] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.664gitlab # [ 4.192282] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.665gitlab # [ 4.193417] systemd[1]: Finished rw-sysroot-nix-store.service.666gitlab # [ 4.194314] systemd[1]: systemd-udevd.service: Deactivated successfully.667gitlab # [ 4.195328] systemd[1]: Stopped Rule-based Manager for Device Events and Files.668gitlab # [ 4.196379] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.669gitlab # [ 4.197391] systemd[1]: Closed udev Control Socket.670gitlab # [ 4.198194] systemd[1]: Starting Cleanup udev Database...671gitlab # [ 4.199209] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.672gitlab # [ 4.200300] systemd[1]: Stopped Create Static Device Nodes in /dev.673gitlab # [ 4.201239] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.674gitlab # [ 4.202400] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.675gitlab # [ 4.203402] systemd[1]: kmod-static-nodes.service: Deactivated successfully.676gitlab # [ 4.204424] systemd[1]: Stopped Create List of Static Device Nodes.677gitlab # [ 4.208281] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.678gitlab # [ 4.209404] systemd[1]: Finished Cleanup udev Database.679gitlab # [ 4.210207] systemd[1]: Reached target Switch Root.680gitlab # [ 4.211881] systemd[1]: Starting NixOS Activation...681gitlab # [ 4.338270] initrd-nixos-activation-start[225]: booting system configuration /nix/store/fpli9vynwlp4k9slgzrmqljc5j0jlrvd-nixos-system-gitlab-test682gitlab # [ 4.384195] initrd-nixos-activation-start[225]: running activation script...683gitlab # [ 4.634276] initrd-nixos-activation-start[248]: setting up /etc...684gitlab # [ 4.723765] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.685gitlab # [ 4.725394] systemd[1]: Finished NixOS Activation.686gitlab # [ 4.727403] systemd[1]: Starting Switch Root...687gitlab # [ 4.753965] systemd[1]: Switching root.688gitlab # [ 4.958573] systemd-journald[90]: Received SIGTERM from PID 1 (systemd).689gitlab # [ 5.109838] NET: Registered PF_VSOCK protocol family690gitlab # [ 5.487320] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)691gitlab # [ 5.497816] systemd[1]: Detected virtualization kvm.692gitlab # [ 5.499787] systemd[1]: Detected architecture x86-64.693gitlab # [ 5.501861] systemd[1]: Detected first boot.694gitlab # [ 5.504403] systemd[1]: Initializing machine ID from random generator.695gitlab # [ 5.647494] systemd[1]: bpf-restrict-fs: LSM BPF program attached696gitlab # [ 5.752864] systemd[1]: Applying preset policy.697gitlab # [ 5.967760] systemd[1]: Populated /etc with preset unit settings.698gitlab # [ 6.095114] systemd[1]: initrd-switch-root.service: Deactivated successfully.699gitlab # [ 6.096518] systemd[1]: Stopped initrd-switch-root.service.700gitlab # [ 6.099537] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.701gitlab # [ 6.101799] systemd[1]: Created slice Slice /system/getty.702gitlab # [ 6.103285] systemd[1]: Created slice GitLab DevOps Platform Slice.703gitlab # [ 6.104932] systemd[1]: Created slice User and Session Slice.704gitlab # [ 6.105927] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.705gitlab # [ 6.107243] systemd[1]: Started Forward Password Requests to Wall Directory Watch.706gitlab # [ 6.108453] systemd[1]: Expecting device /dev/hvc0...707gitlab # [ 6.109326] systemd[1]: Expecting device /dev/ttyS0...708gitlab # [ 6.110196] systemd[1]: Reached target Local Encrypted Volumes.709gitlab # [ 6.111138] systemd[1]: Stopped target initrd-fs.target.710gitlab # [ 6.111980] systemd[1]: Stopped target initrd-root-fs.target.711gitlab # [ 6.112864] systemd[1]: Stopped target initrd-switch-root.target.712gitlab # [ 6.113789] systemd[1]: Reached target Virtual Machines and Containers.713gitlab # [ 6.114804] systemd[1]: Reached target Path Units.714gitlab # [ 6.115582] systemd[1]: Reached target Remote File Systems.715gitlab # [ 6.116494] systemd[1]: Reached target Slice Units.716gitlab # [ 6.117372] systemd[1]: Reached target Swaps.717gitlab # [ 6.118965] systemd[1]: Listening on Query the User Interactively for a Password.718gitlab # [ 6.121148] systemd[1]: Listening on Process Core Dump Socket.719gitlab # [ 6.122606] systemd[1]: Listening on Credential Encryption/Decryption.720gitlab # [ 6.124301] systemd[1]: Listening on Factory Reset Management.721gitlab # [ 6.125335] systemd[1]: Listening on Hostname Service Socket.722gitlab # [ 6.127938] systemd[1]: Starting Journal Log Access Socket...723gitlab # [ 6.129575] systemd[1]: Listening on Journal Audit Socket.724gitlab # [ 6.131615] systemd[1]: Listening on Console Output Muting Service Socket.725gitlab # [ 6.132926] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.726gitlab # [ 6.134284] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os727gitlab # [ 6.135853] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki728gitlab # [ 6.138576] systemd[1]: Listening on Disk Repartitioning Service Socket.729gitlab # [ 6.139767] systemd[1]: Listening on udev Control Socket.730gitlab # [ 6.140785] systemd[1]: Listening on udev Varlink Socket.731gitlab # [ 6.144218] systemd[1]: Mounting Huge Pages File System...732gitlab # [ 6.146527] systemd[1]: Mounting POSIX Message Queue File System...733gitlab # [ 6.148815] systemd[1]: Mounting Kernel Debug File System...734gitlab # [ 6.150793] systemd[1]: Mounting Kernel Trace File System...735gitlab # [ 6.153991] systemd[1]: Starting Create List of Static Device Nodes...736gitlab # [ 6.156793] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio737gitlab # [ 6.165444] systemd[1]: Starting Load Kernel Module configfs...738gitlab # [ 6.171049] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm739gitlab # [ 6.174833] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore740gitlab # [ 6.208801] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse741gitlab # [ 6.214030] systemd[1]: Mounting FUSE Control File System...742gitlab # [ 6.215935] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67743gitlab # [ 6.223835] systemd[1]: Starting Journal Service...744gitlab # [ 6.227488] systemd[1]: Starting Load Kernel Modules...745gitlab # [ 6.233408] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...746gitlab # [ 6.235525] systemd[1]: Starting Remount Root and Kernel File Systems...747gitlab # [ 6.236523] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os748gitlab # [ 6.240365] systemd[1]: Starting Coldplug All udev Devices...749gitlab # [ 6.246337] systemd[1]: Listening on Journal Log Access Socket.750gitlab # [ 6.247452] systemd[1]: Mounted Huge Pages File System.751gitlab # [ 6.247607] systemd-journald[318]: Collecting audit messages is enabled.752gitlab # [ 6.247855] systemd[1]: Mounted POSIX Message Queue File System.753gitlab # [ 6.250392] systemd[1]: Mounted Kernel Debug File System.754gitlab # [ 6.251558] systemd[1]: Mounted Kernel Trace File System.755gitlab # [ 6.253245] systemd[1]: Finished Create List of Static Device Nodes.756gitlab # [ 6.255209] systemd[1]: modprobe@configfs.service: Deactivated successfully.757gitlab # [ 6.257326] systemd[1]: Finished Load Kernel Module configfs.758gitlab # [ 6.259069] systemd[1]: Mounted FUSE Control File System.759gitlab # [ 6.262828] systemd[1]: Mounting Kernel Configuration File System...760gitlab # [ 6.264422] loop: module loaded761gitlab # [ 6.118325] systemd[1]: Queued start job for default target Multi-User System.762gitlab # [ 6.119630] systemd[1]: Unnecessary job was r[ 6.267314] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...763gitlab # emoved for /dev/vdb.764gitlab # [ 6.268868] systemd[1]: Started Journal Service.765gitlab # [ 6.121683] systemd[1]: systemd-journald.service: Deactivated successfully.766gitlab # [ 6.123494] systemd-modules-load[319]: Using 3 probe threads767gitlab # [ 6.124597] systemd-modules-load[319]: Inserted module 'tls'768gitlab # [ 6.125729] systemd-modules-load[319]: Inserted module 'loop'769gitlab # [ 6.127134] systemd[1]: Finis[ 6.274678] EXT4-fs (vda): re-mounted a14ca34f-8c5a-480e-931e-7cf7c398cd60.770gitlab # hed Load Kernel Modules.771gitlab # [ 6.136073] systemd[1]: Starting Firewall...772gitlab # [ 6.136978] systemd[1]: Starting Apply Kernel Variables...773gitlab # [ 6.141181] systemd[1]: Finished Remount Root and Kernel File Systems.774gitlab # [ 6.142476] systemd[1]: Mounted Kernel Configuration File System.775gitlab # [ 6.149562] systemd-oomd[321]: No swap; memory pressure usage will be degraded776gitlab # [ 6.154281] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.777gitlab # [ 6.155452] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.778gitlab # [ 6.156527] systemd[1]: Listening on Disk Image Download Service Socket.779gitlab # [ 6.157640] systemd[1]: Starting Flush Journal to Persistent Storage...780gitlab # [ 6.158729] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore781gitlab # [ 6.160318] systemd[1]: Starting Load/Save OS Random Seed...782gitlab # [ 6.162270] systemd[1]: Starting Create Static Device Nodes in /dev...783gitlab # [ 6.163460] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os784gitlab # [ 6.171127] systemd[1]: Finished Apply Kernel Variables.785gitlab # [ 6.323433] systemd-journald[318]: Received client request to flush runtime journal.786gitlab # [ 6.225901] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 6.227767] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 6.228807] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 6.230299] systemd[1]: Finished Load/Save OS Random Seed.790gitlab # [ 6.231288] systemd[1]: Reached target First Boot Complete.791gitlab # [ 6.232414] systemd-udevd[353]: Using default interface naming scheme 'v261'.792gitlab # [ 6.233515] systemd[1]: Finished Flush Journal to Persistent Storage.793gitlab # [ 6.247499] systemd[1]: Finished Coldplug All udev Devices.794gitlab # [ 6.255469] systemd[1]: Started Rule-based Manager for Device Events and Files.795gitlab # [ 6.326519] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse796gitlab # [ 6.346149] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.797gitlab # [ 6.359733] (udev-worker)[419]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.798gitlab # [ 6.363737] (udev-worker)[419]: Network interface NamePolicy= disabled on kernel command line.799gitlab # [ 6.365373] (udev-worker)[413]: Network interface NamePolicy= disabled on kernel command line.800gitlab # [ 6.369089] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.801gitlab # [ 6.389875] systemd[1]: Condition check resulted in Virtio network device being skipped.802gitlab # [ 6.391207] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore803gitlab # [ 6.392885] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67804gitlab # [ 6.395613] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore805gitlab # [ 6.398447] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os806gitlab # [ 6.400276] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os807gitlab # [ 6.561443] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3808gitlab # [ 6.569074] ACPI: button: Power Button [PWRF]809gitlab # [ 6.424726] systemd[1]: Finished Firewall.810gitlab # [ 6.594132] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4811gitlab # [ 6.596199] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console812gitlab # [ 6.596332] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0813gitlab # [ 6.598786] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD814gitlab # [ 6.610820] Floppy drive(s): fd0 is 2.88M AMI BIOS815gitlab # [ 6.621012] Console: switching to colour dummy device 80x25816gitlab # [ 6.623520] [drm] Found bochs VGA, ID 0xb0c5.817gitlab # [ 6.623523] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.818gitlab # [ 6.624495] parport_pc 00:03: reported by Plug and Play ACPI819gitlab # [ 6.625400] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic820gitlab # [ 6.626132] parport0: PC-style at 0x378821gitlab # [ 6.626763] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0822gitlab # [ 6.626766] , irq 7823gitlab # [ 6.626767] [PCSPP(,...)]824gitlab # [ 6.627346] FDC 0 is a S82078B825gitlab # [ 6.627898] rtc_cmos 00:05: RTC can wake from S4826gitlab # [ 6.638451] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6827gitlab # [ 6.639397] rtc_cmos 00:05: registered as rtc0828gitlab # [ 6.639484] rtc_cmos 00:05: setting system clock to 2026-10-04T18:56:21 UTC (1791140181)829gitlab # [ 6.639603] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs830gitlab # [ 6.640016] Console: switching to colour frame buffer device 160x50831gitlab # [ 6.640884] systemd-journald[318]: Time jumped backwards, rotating.832gitlab # [ 6.641804] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device833gitlab # [ 6.646132] mousedev: PS/2 mouse device common for all mice834gitlab # [ 6.647362] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5835gitlab # [ 6.547165] systemd[1]: Starting Virtual Console Setup...836gitlab # [ 6.709767] ppdev: user-space parallel port driver837gitlab # [ 6.584443] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.838gitlab # [ 6.586332] systemd[1]: Stopped Virtual Console Setup.839gitlab # [ 6.591173] systemd[1]: Starting Virtual Console Setup...840gitlab # [ 6.595394] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.841gitlab # [ 6.598148] systemd[1]: Stopped Virtual Console Setup.842gitlab # [ 6.621595] systemd[1]: Starting Virtual Console Setup...843gitlab # [ 6.810235] kvm_amd: TSC scaling supported844gitlab # [ 6.810914] kvm_amd: Nested Virtualization enabled845gitlab # [ 6.811542] kvm_amd: Nested Paging enabled846gitlab # [ 6.812222] kvm_amd: LBR virtualization supported847gitlab # [ 6.812930] kvm_amd: Virtual VMLOAD VMSAVE supported848gitlab # [ 6.813587] kvm_amd: Virtual GIF supported849gitlab # [ 6.814211] kvm_amd: Virtual NMI enabled850gitlab # [ 6.689545] systemd-vconsole-setup[496]: Configuration of first virtual console was skipped, ignoring remaining ones.851gitlab # [ 6.692692] systemd[1]: Finished Virtual Console Setup.852gitlab # [ 6.849960] EDAC MC: Ver: 3.0.0853gitlab # [ 6.952720] systemd[1]: Mounting /run/wrappers...854gitlab # [ 6.978637] systemd[1]: Mounted /run/wrappers.855gitlab # [ 6.981611] systemd[1]: Reached target Local File Systems.856gitlab # [ 6.984987] systemd[1]: Listening on Boot Loader Control Service Socket.857gitlab # [ 6.989632] systemd[1]: Starting register-nix-paths.service...858gitlab # [ 6.994072] systemd[1]: Starting Create SUID/SGID Wrappers...859gitlab # [ 6.996318] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.860gitlab # [ 7.003638] systemd[1]: Starting Save Transient machine-id to Disk...861gitlab # [ 7.009697] systemd[1]: Starting Create System Files and Directories...862gitlab # [ 7.044200] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.863gitlab # [ 7.049218] systemd[1]: Finished Save Transient machine-id to Disk.864gitlab # [ 7.054363] systemd[1]: Finished Create System Files and Directories.865gitlab # [ 7.061467] systemd[1]: Starting Rebuild Journal Catalog...866gitlab # [ 7.063537] systemd[1]: Starting Record System Boot/Shutdown in UTMP...867gitlab # [ 7.080459] systemd[1]: Finished Record System Boot/Shutdown in UTMP.868gitlab # [ 7.098469] systemd[1]: Finished Rebuild Journal Catalog.869gitlab # [ 7.100225] systemd[1]: Starting Update is Completed...870gitlab # [ 7.115280] systemd[1]: Finished Update is Completed.871gitlab # [ 7.154269] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.872gitlab # [ 7.155408] systemd[1]: Finished Create SUID/SGID Wrappers.873gitlab # [ 7.435225] systemd[1]: Finished register-nix-paths.service.874gitlab # [ 7.436545] systemd[1]: Reached target System Initialization.875gitlab # [ 7.437692] systemd[1]: Started Discard unused filesystem blocks once a week.876gitlab # [ 7.439097] systemd[1]: Started logrotate.timer.877gitlab # [ 7.440174] systemd[1]: Started Daily Cleanup of Temporary Directories.878gitlab # [ 7.441433] systemd[1]: Reached target Timer Units.879gitlab # [ 7.442399] systemd[1]: Listening on D-Bus System Message Bus Socket.880gitlab # [ 7.443572] systemd[1]: Listening on Nix Daemon Socket.881gitlab # [ 7.448655] systemd[1]: Starting nixbot CI service socket...882gitlab # [ 7.449768] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.883gitlab # [ 7.451278] systemd[1]: Listening on nixbot CI service socket.884gitlab # [ 7.452418] systemd[1]: Reached target Socket Units.885gitlab # [ 7.454777] systemd[1]: Starting D-Bus System Message Bus...886gitlab # [ 7.496730] dbus-broker-launch[578]: Looking up NSS user entry for 'systemd-timesync'...887gitlab # [ 7.498381] dbus-broker-launch[578]: NSS returned no entry for 'systemd-timesync'888gitlab # [ 7.499707] dbus-broker-launch[578]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"889gitlab # [ 7.505750] systemd[1]: Started D-Bus System Message Bus.890gitlab # [ 7.523506] dbus-broker-launch[578]: Ready891gitlab # [ 7.525532] systemd[1]: Reached target Basic System.892gitlab # [ 7.528450] systemd[1]: Started backdoor.service.893gitlab # [ 7.531303] systemd[1]: Starting gitlab-config.service...894gitlab # [ 7.534079] systemd[1]: Starting Import lastlog data into lastlog2 database...895gitlab # [ 7.546195] systemd[1]: Starting Logrotate configuration check...896gitlab # [ 7.549435] systemd[1]: Starting Name Service Cache Daemon (nsncd)...897gitlab # [ 7.556401] systemd[1]: Starting Post-Boot Actions...898gitlab # [ 7.558633] logrotate[583]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.899gitlab # [ 7.561353] logrotate[583]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf900gitlab # [ 7.564245] logrotate[583]: note: 'monthly' overrides previously specified 'weekly'901gitlab # [ 7.565603] logrotate[583]: Reading state from file: /var/lib/logrotate.status902gitlab # [ 7.566804] logrotate[583]: state file /var/lib/logrotate.status does not exist903gitlab # [ 7.567906] logrotate[583]: Allocating hash table for state file, size 64 entries904gitlab # connecting to host...905gitlab # [ 7.569772] logrotate[583]: Handling 4 logs906gitlab # [ 7.570277] logrotate[583]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed907gitlab # [ 7.570841] logrotate[583]: considering log /var/log/btmp908gitlab # [ 7.574176] logrotate[583]: Creating new state909gitlab # [ 7.574715] logrotate[583]: Now: 2026-10-04 18:56910gitlab # [ 7.575486] logrotate[583]: Last rotated at 2026-10-04 18:00911gitlab # [ 7.579194] logrotate[583]: log does not need rotating (log has already been rotated)912gitlab # [ 7.579737] logrotate[583]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed913gitlab # [ 7.580660] logrotate[583]: considering log /var/log/wtmp914gitlab # [ 7.581542] logrotate[583]: Creating new state915gitlab # [ 7.581926] logrotate[583]: Now: 2026-10-04 18:56916gitlab # [ 7.585603] logrotate[583]: Last rotated at 2026-10-04 18:00917gitlab: Guest shell says: b'Spawning backdoor root shell...\n'918gitlab # [ 7.586275] logrotate[583]: log does not need rotating (log has already been rotated)919gitlab # [ 7.586982] logrotate[583]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed920gitlab: connected to guest root shell921gitlab # [ 7.587605] logrotate[583]: switching euid from 0 to 165 and egid from 0 to 165 (pid 583)922gitlab: (connecting took 8.24 seconds)923gitlab # [ 7.592091] logrotate[583]: considering log /var/gitlab/state/log/*.log924gitlab: (finished: waiting for the VM to finish booting, in 8.24 seconds)925gitlab # [ 7.592321] logrotate[583]: log /var/gitlab/state/log/*.log does not exist -- skipping926gitlab # [ 7.592782] logrotate[583]: Creating new state927gitlab # [ 7.593384] logrotate[583]: switching euid from 165 to 0 and egid from 165 to 0 (pid 583)928gitlab # [ 7.593838] logrotate[583]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed929gitlab # [ 7.594450] logrotate[583]: switching euid from 0 to 60 and egid from 0 to 60 (pid 583)930gitlab # [ 7.594887] logrotate[583]: considering log /var/log/nginx/*.log931gitlab # [ 7.595480] logrotate[583]: log /var/log/nginx/*.log does not exist -- skipping932gitlab # [ 7.595904] logrotate[583]: Creating new state933gitlab # [ 7.596506] logrotate[583]: not running postrotate script, since no logs were rotated934gitlab # [ 7.596933] logrotate[583]: switching euid from 60 to 0 and egid from 60 to 0 (pid 583)935gitlab # [ 7.597740] systemd[1]: Started Reset console on configuration changes.936gitlab # [ 7.598328] systemd[1]: Starting resolvconf update...937gitlab # [ 7.598860] systemd[1]: Finished Logrotate configuration check.938gitlab # [ 7.604881] systemd[1]: Finished Post-Boot Actions.939gitlab # [ 7.615134] systemd[1]: Finished Import lastlog data into lastlog2 database.940gitlab # [ 7.624728] systemd[1]: Started Name Service Cache Daemon (nsncd).941gitlab # [ 7.626558] nsncd[584]: Oct 04 18:56:22.632 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"942gitlab # [ 7.626938] systemd[1]: Reached target Host and Network Name Lookups.943gitlab # [ 7.629182] systemd[1]: Reached target User and Group Name Lookups.944gitlab # [ 7.633473] systemd[1]: Starting User Login Management...945gitlab # [ 7.666087] systemd[1]: Stopped target Host and Network Name Lookups.946gitlab # [ 7.666296] systemd[1]: Stopping Host and Network Name Lookups...947gitlab # [ 7.668389] systemd[1]: Stopped target User and Group Name Lookups.948gitlab # [ 7.668759] systemd[1]: Stopping User and Group Name Lookups...949gitlab # [ 7.668979] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...950gitlab # [ 7.673529] systemd[1]: nscd.service: Deactivated successfully.951gitlab # [ 7.674617] systemd[1]: Stopped Name Service Cache Daemon (nsncd).952gitlab # [ 7.676355] systemd[1]: nscd.service: Consumed 27ms CPU time over 123ms wall clock time, 7.4M memory peak, 5.5M read from disk.953gitlab # [ 7.678882] systemd-logind[628]: New seat seat0.954gitlab # [ 7.680384] systemd-logind[628]: Watching system buttons on /dev/input/event2 (Power Button)955gitlab # [ 7.682117] systemd-logind[628]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)956gitlab # [ 7.683470] systemd-logind[628]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)957gitlab # [ 7.686167] systemd[1]: Started User Login Management.958gitlab # [ 7.686938] systemd[1]: Starting linger-users.service...959gitlab # [ 7.688278] systemd[1]: Starting Name Service Cache Daemon (nsncd)...960gitlab # [ 7.698324] systemd[1]: Finished resolvconf update.961gitlab # [ 7.699428] systemd[1]: Reached target Preparation for Network.962gitlab # [ 7.702210] systemd[1]: Starting DHCP Client...963gitlab # [ 7.703651] systemd[1]: Starting Address configuration of eth1...964gitlab # [ 7.707090] systemd[1]: Starting Extra networking commands....965gitlab # [ 7.708171] nsncd[682]: Oct 04 18:56:22.714 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"966gitlab # [ 7.708533] systemd[1]: Started Name Service Cache Daemon (nsncd).967gitlab # [ 7.709067] systemd[1]: linger-users.service: Deactivated successfully.968gitlab # [ 7.712327] systemd[1]: Finished linger-users.service.969gitlab # [ 7.714719] systemd[1]: Reached target Host and Network Name Lookups.970gitlab # [ 7.717648] systemd[1]: Reached target User and Group Name Lookups.971gitlab # [ 7.740367] network-addresses-eth1-start[690]: adding address 192.168.1.1/24... done972gitlab # [ 7.742575] network-addresses-eth1-start[690]: adding address 2001:db8:1::1/64... done973gitlab # [ 7.747843] systemd[1]: Finished Address configuration of eth1.974gitlab # [ 7.771783] dhcpcd[708]: dhcpcd-10.3.2 starting975gitlab # [ 7.774787] dhcpcd[764]: dev: loaded udev976gitlab # [ 7.776071] systemd[1]: Finished Extra networking commands..977gitlab # [ 7.776552] systemd[1]: Reached target Network.978gitlab # [ 7.777804] systemd[1]: Starting gitlab-workhorse.service...979gitlab # [ 7.929590] 8021q: 802.1Q VLAN Support v1.8980gitlab # [ 7.782208] systemd[1]: Starting Nginx Web Server...981gitlab # [ 7.930936] 8021q: adding VLAN 0 to HW filter on device eth1982gitlab # [ 7.784720] systemd[1]: Starting PostgreSQL Server...983gitlab # [ 7.789217] systemd[1]: Starting Redis Server - redis-gitlab...984gitlab # [ 7.794146] systemd[1]: Starting Permit User Sessions...985gitlab # [ 7.801080] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.986gitlab # [ 7.809286] systemd[1]: Finished Permit User Sessions.987gitlab # [ 7.819245] systemd[1]: Started Getty on tty1.988gitlab # [ 7.819745] systemd[1]: Reached target Login Prompts.989gitlab # [ 8.010266] cfg80211: Loading compiled-in X.509 certificates for regulatory database990gitlab # [ 8.019398] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'991gitlab # [ 8.020919] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'992gitlab # [ 8.022531] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2993gitlab # [ 8.023677] cfg80211: failed to load regulatory.db994gitlab # [ 8.057548] 8021q: adding VLAN 0 to HW filter on device eth0995gitlab # [ 7.913451] dhcpcd[764]: eth0: waiting for carrier996gitlab # [ 7.914722] dhcpcd[764]: eth0: carrier acquired997gitlab # [ 7.918422] dhcpcd[764]: DUID 00:01:00:01:32:55:5d:d6:52:54:00:12:34:56998gitlab # [ 7.919441] dhcpcd[764]: eth0: IAID 00:12:34:56999gitlab # [ 7.920220] dhcpcd[764]: eth0: adding address fe80::5054:ff:fe12:34561000gitlab # [ 7.936303] redis-server[790]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1001gitlab # [ 7.938653] redis-server[790]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1002gitlab # [ 7.940313] redis-server[790]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=790, just started1003gitlab # [ 7.942030] systemd[1]: Started Redis Server - redis-gitlab.1004gitlab # [ 7.942929] redis-server[790]: Configuration loaded1005gitlab # [ 7.943831] redis[790]: monotonic clock: POSIX clock_gettime1006gitlab # [ 7.944955] redis[790]: Running mode=standalone, port=0.1007gitlab # [ 7.946081] redis[790]: Server initialized1008gitlab # [ 7.947109] nginx-pre-start[792]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok1009gitlab # [ 7.949306] nginx-pre-start[792]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1010gitlab # [ 7.951227] redis[790]: Ready to accept connections unix1011gitlab # [ 7.952229] systemd[1]: Started Nginx Web Server.1012gitlab # [ 7.953331] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres".1013gitlab # [ 7.955143] postgresql-pre-start[794]: This user must also own the server process.1014gitlab # [ 7.956530] redis[790]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1015gitlab # [ 7.961274] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8".1016gitlab # [ 7.961889] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8".1017gitlab # [ 7.962631] postgresql-pre-start[794]: The default text search configuration will be set to "english".1018gitlab # [ 7.963146] postgresql-pre-start[794]: Data page checksums are enabled.1019gitlab # [ 7.963528] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1020gitlab # [ 7.967271] postgresql-pre-start[794]: creating subdirectories ... ok1021gitlab # [ 7.967754] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix1022gitlab # [ 8.000787] postgresql-pre-start[794]: selecting default "max_connections" ... 1001023gitlab # [ 8.038682] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB1024gitlab # [ 8.341694] systemd[1]: Started gitlab-workhorse.service.1025gitlab # [ 8.438699] postgresql-pre-start[794]: selecting default time zone ... UTC1026gitlab # [ 8.440604] postgresql-pre-start[794]: creating configuration files ... ok1027gitlab # [ 8.594881] gitlab-workhorse[826]: time="2026-10-04T18:56:23Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41028gitlab # [ 8.595275] gitlab-workhorse[826]: time="2026-10-04T18:56:23Z" level=info msg="Using redis/go-redis"1029gitlab # [ 8.595867] gitlab-workhorse[826]: time="2026-10-04T18:56:23Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1030gitlab # [ 8.611262] postgresql-pre-start[794]: running bootstrap script ... ok1031gitlab # [ 9.055668] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok1032gitlab # [ 9.149301] dhcpcd[764]: eth0: soliciting a DHCP lease1033gitlab # [ 9.309903] NET: Registered PF_PACKET protocol family1034gitlab # [ 9.176136] dhcpcd[764]: eth0: offered 10.0.2.15 from 10.0.2.21035gitlab # [ 9.182426] dhcpcd[764]: eth0: probing address 10.0.2.15/241036gitlab # [ 9.350127] postgresql-pre-start[794]: syncing data to disk ... ok1037gitlab # [ 9.352623] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections1038gitlab # [ 9.355775] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1039gitlab # [ 9.360989] postgresql-pre-start[794]: Success. You can now start the database server using:1040gitlab # [ 9.363832] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1041gitlab # [ 9.435818] dhcpcd[764]: eth0: soliciting an IPv6 router1042gitlab # [ 9.437911] dhcpcd[764]: eth0: Router Advertisement from fe80::21043gitlab # [ 9.440197] dhcpcd[764]: eth0: adding address fec0::5054:ff:fe12:3456/641044gitlab # [ 9.440463] dhcpcd[764]: eth0: adding route to fec0::/641045gitlab # [ 9.441778] dhcpcd[764]: eth0: adding default route via fe80::21046gitlab # [ 9.478589] postgres[852]: [852] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1047gitlab # [ 9.482838] postgres[852]: [852] LOG: listening on IPv6 address "::1", port 54321048gitlab # [ 9.483648] postgres[852]: [852] LOG: listening on IPv4 address "127.0.0.1", port 54321049gitlab # [ 9.492428] postgres[852]: [852] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1050gitlab # [ 9.511523] postgres[862]: [862] LOG: database system was shut down at 2026-10-04 18:56:24 GMT1051gitlab # [ 9.527100] postgres[852]: [852] LOG: database system is ready to accept connections1052gitlab # [ 9.535890] systemd[1]: Started PostgreSQL Server.1053gitlab # [ 9.538325] systemd[1]: Starting PostgreSQL Setup Scripts...1054gitlab # [ 9.766422] postgresql-setup-start[878]: CREATE DATABASE1055gitlab # [ 9.795323] postgresql-setup-start[892]: CREATE ROLE1056gitlab # [ 9.797734] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[871]: mkdir -p /var/gitlab/state/home/.ssh: OK1057gitlab # [ 9.803215] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[871]: chmod 700 /var/gitlab/state/home/.ssh: OK1058gitlab # [ 9.808543] postgresql-setup-start[895]: ALTER DATABASE1059gitlab # [ 9.831637] postgresql-setup-start[906]: CREATE ROLE1060gitlab # [ 9.834313] systemd[1]: Finished PostgreSQL Setup Scripts.1061gitlab # [ 9.836127] systemd[1]: Reached target PostgreSQL.1062gitlab # [ 9.838698] systemd[1]: Starting gitlab-postgresql.service...1063gitlab # [ 9.851129] systemd[1]: Finished gitlab-config.service.1064gitlab # [ 9.853095] systemd[1]: Started gitaly.service.1065gitlab # [ 9.856075] systemd[1]: Starting gitlab-db-config.service...1066gitlab # [ 9.940678] gitlab-postgresql-start[922]: CREATE DATABASE1067gitlab # [ 9.992209] gitlab-postgresql-start[928]: CREATE EXTENSION1068gitlab # [ 10.092810] gitlab-postgresql-start[930]: CREATE EXTENSION1069gitlab # [ 10.095968] systemd[1]: Finished gitlab-postgresql.service.1070gitlab # [ 10.407169] gitaly[915]: time="2026-10-04T18:56:25.414Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9151071gitlab # [ 10.415947] gitaly[915]: time="2026-10-04T18:56:25.414Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9151072gitlab # [ 10.419769] gitaly[915]: time="2026-10-04T18:56:25.414Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9151073gitlab # [ 10.423747] gitaly[915]: time="2026-10-04T18:56:25.414Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9151074gitlab # [ 10.427343] gitaly[915]: time="2026-10-04T18:56:25.414Z" level=info msg="Starting Gitaly" pid=915 version=1075gitlab # [ 10.599924] gitaly[915]: time="2026-10-04T18:56:25.607Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=192 pid=9151076gitlab # [ 10.601915] gitaly[915]: time="2026-10-04T18:56:25.608Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9151077gitlab # [ 10.603518] gitaly[915]: time="2026-10-04T18:56:25.608Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9151078gitlab # [ 10.605263] gitaly[915]: time="2026-10-04T18:56:25.608Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-915/git-exec-3063103681.d/git pid=9151079gitlab # [ 10.607263] gitaly[915]: time="2026-10-04T18:56:25.612Z" level=info msg="finished detecting git version" duration_ms=4 pid=9151080gitlab # [ 10.608974] gitaly[915]: time="2026-10-04T18:56:25.612Z" level=info msg="using Git version" pid=915 version=2.53.g1081gitlab # [ 10.610496] gitaly[915]: time="2026-10-04T18:56:25.613Z" level=info msg="clearing disk cache object folder" pid=915 storage=default1082gitlab # [ 10.612254] gitaly[915]: time="2026-10-04T18:56:25.613Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1838574654 pid=915 storage=default1083gitlab # [ 10.615361] gitaly[915]: time="2026-10-04T18:56:25.613Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1838574654 pid=915 storage=default1084gitlab # [ 10.618375] gitaly[915]: time="2026-10-04T18:56:25.613Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9151085gitlab # [ 10.621182] gitaly[915]: time="2026-10-04T18:56:25.613Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9151086gitlab # [ 10.623955] gitaly[915]: time="2026-10-04T18:56:25.613Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1838574654 clear_duration_ms=0 pid=915 storage=default1087gitlab # [ 10.641741] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9151088gitlab # [ 10.644299] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=915 schema=unix1089gitlab # [ 10.644630] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="finished tempdir cleaner walk" pid=915 storage=default time_ms=01090gitlab # [ 10.645510] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="listening at address" address=/run/gitaly/gitaly-915/sock.d/intern pid=915 schema=unix1091gitlab # [ 10.646116] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="counting repositories" pid=9151092gitlab # [ 10.646621] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="starting to count repositories" pid=915 storage_path=/var/gitlab/state/repositories1093gitlab # [ 10.647073] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=915 storage_path=/var/gitlab/state/repositories1094gitlab # [ 10.647636] gitaly[915]: time="2026-10-04T18:56:25.649Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9151095gitlab # [ 10.648101] gitaly[915]: time="2026-10-04T18:56:25.650Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9151096gitlab # [ 10.648453] gitaly[915]: time="2026-10-04T18:56:25.650Z" level=info msg="Started Gitaly" duration_ms=236 pid=9151097gitlab # [ 10.649108] gitaly[915]: time="2026-10-04T18:56:25.651Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9151098gitlab # [ 10.649723] gitaly[915]: time="2026-10-04T18:56:25.651Z" level=info msg="maintenance: daily scheduled" pid=915 scheduled="2026-10-05 12:00:00 +0000 UTC"1099gitlab # [ 11.659762] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1100gitlab # [ 11.660964] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1101gitlab # [ 11.705111] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1102gitlab # [ 12.659471] gitaly[915]: time="2026-10-04T18:56:27.666Z" level=info msg="License database preloaded" duration_ms=2251 pid=9151103gitlab # [ 14.844693] dhcpcd[764]: eth0: leased 10.0.2.15 for 86400 seconds1104gitlab # [ 14.845162] dhcpcd[764]: eth0: adding route to 10.0.2.0/241105gitlab # [ 14.845920] dhcpcd[764]: eth0: adding default route via 10.0.2.21106gitlab # [ 14.968751] systemd[1]: Started DHCP Client.1107gitlab # [ 14.970293] systemd[1]: Reached target Network is Online.1108gitlab # [ 43.315201] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[923]: W, [2026-10-04T18:56:58.322401 #923] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1109gitlab # [ 45.474276] postgres[1330]: [1330] ERROR: relation "feature_gates" does not exist at character 5231110gitlab # [ 45.474623] postgres[1330]: [1330] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1111gitlab # [ 45.476865] postgres[1330]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1112gitlab # [ 45.479510] postgres[1330]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1113gitlab # [ 45.482313] postgres[1330]: attidentity AS identity,1114gitlab # [ 45.484605] postgres[1330]: attgenerated as attgenerated1115gitlab # [ 45.487290] postgres[1330]: FROM pg_attribute a1116gitlab # [ 45.489520] postgres[1330]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1117gitlab # [ 45.492343] postgres[1330]: LEFT JOIN pg_type t ON a.atttypid = t.oid1118gitlab # [ 45.494524] postgres[1330]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1119gitlab # [ 45.497214] postgres[1330]: WHERE a.attrelid = '"feature_gates"'::regclass1120gitlab # [ 45.499447] postgres[1330]: AND a.attnum > 0 AND NOT a.attisdropped1121gitlab # [ 45.501378] postgres[1330]: ORDER BY a.attnum1122gitlab # [ 45.503210] postgres[1330]:1123gitlab # [ 48.651280] gitlab-workhorse[826]: time="2026-10-04T18:57:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1124gitlab # [ 49.175580] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1372]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1125gitlab # [ 49.180439] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1372]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1126gitlab # [ 58.669337] gitlab-workhorse[826]: time="2026-10-04T18:57:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1127gitlab # [ 64.976591] postgres[1447]: [1447] ERROR: relation "table_without_models" does not exist1128gitlab # [ 64.976997] postgres[1447]: [1447] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1129gitlab # [ 68.678911] gitlab-workhorse[826]: time="2026-10-04T18:57:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1130gitlab # [ 78.092933] systemd[1]: Finished gitlab-db-config.service.1131gitlab # [ 78.096815] systemd[1]: Started gitlab-sidekiq.service.1132gitlab # [ 78.100344] systemd[1]: Starting gitlab.service...1133gitlab # [ 78.462528] bundle[1632]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1134gitlab # [ 78.467341] bundle[1632]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1135gitlab # [ 78.471955] bundle[1632]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1136gitlab # [ 78.685457] gitlab-workhorse[826]: time="2026-10-04T18:57:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1137gitlab # [ 81.371124] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.357Z","pid":1632,"message":"* Enabling systemd notification integration"}1138gitlab # [ 81.374943] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"Puma starting in cluster mode..."}1139gitlab # [ 81.378724] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1140gitlab # [ 81.382558] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1141gitlab # [ 81.386933] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Min threads: 0"}1142gitlab # [ 81.390140] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Max threads: 4"}1143gitlab # [ 81.393209] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Environment: production"}1144gitlab # [ 81.396457] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Master PID: 1632"}1145gitlab # [ 81.399534] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Workers: 2"}1146gitlab # [ 81.402503] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1147gitlab # [ 81.406783] bundle[1632]: {"timestamp":"2026-10-04T18:57:36.378Z","pid":1632,"message":"* Preloading application"}1148gitlab # [ 81.410671] sidekiq-cluster[1631]: {"severity":"INFO","time":"2026-10-04T18:57:36.364Z","message":"Starting cluster with 1 processes"}1149gitlab # [ 88.694923] gitlab-workhorse[826]: time="2026-10-04T18:57:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1150gitlab # [ 98.697155] gitlab-workhorse[826]: time="2026-10-04T18:57:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1151gitlab # [ 105.855708] bundle[1632]: W, [2026-10-04T18:58:00.862847 #1632] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1152gitlab # [ 107.340746] sidekiq-cluster[1662]: 2026-10-04T18:58:02.348Z pid=1662 tid=wk6 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1662, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430718976}1153gitlab # [ 107.515309] sidekiq-cluster[1662]: W, [2026-10-04T18:58:02.523173 #1662] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1154gitlab # [ 107.554145] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:02.561Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1155gitlab # [ 108.705867] gitlab-workhorse[826]: time="2026-10-04T18:58:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1156gitlab # [ 109.505772] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.512Z","message":"GitLab reliable fetch activated!"}1157gitlab # [ 109.506596] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.513Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1158gitlab # [ 109.538616] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.546Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1159gitlab # [ 109.543997] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.551Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1160gitlab # [ 109.549750] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.557Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1161gitlab # [ 109.555040] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.563Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1162gitlab # [ 109.560216] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.568Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1163gitlab # [ 109.564805] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.572Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1164gitlab # [ 109.569355] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.577Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1165gitlab # [ 109.573366] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.581Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1166gitlab # [ 109.576279] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.584Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1167gitlab # [ 109.579341] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.587Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1168gitlab # [ 109.581218] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.589Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1169gitlab # [ 109.583583] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.591Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1170gitlab # [ 109.585407] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.593Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1171gitlab # [ 109.588841] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.596Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1172gitlab # [ 109.591616] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.599Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1173gitlab # [ 109.593918] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.601Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1174gitlab # [ 109.596565] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.604Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1175gitlab # [ 109.599646] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.607Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1176gitlab # [ 109.607384] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.615Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1177gitlab # [ 109.611115] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.619Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1178gitlab # [ 109.613408] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.621Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1179gitlab # [ 109.615279] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.623Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1180gitlab # [ 109.618435] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.626Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1181gitlab # [ 109.621063] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.629Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1182gitlab # [ 109.623392] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.631Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1183gitlab # [ 109.625385] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.633Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1184gitlab # [ 109.627937] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.635Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1185gitlab # [ 109.629923] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.637Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1186gitlab # [ 109.633882] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.641Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1187gitlab # [ 109.637253] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.645Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1188gitlab # [ 109.640851] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.648Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1189gitlab # [ 109.642885] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.650Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1190gitlab # [ 109.646068] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.654Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1191gitlab # [ 109.650320] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.658Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1192gitlab # [ 109.652137] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.660Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1193gitlab # [ 109.653673] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.661Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1194gitlab # [ 109.655280] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.663Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1195gitlab # [ 109.656988] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.665Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1196gitlab # [ 109.658659] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.666Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1197gitlab # [ 109.660288] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.668Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1198gitlab # [ 109.662167] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.670Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1199gitlab # [ 109.663767] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.671Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1200gitlab # [ 109.665660] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.673Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1201gitlab # [ 109.667539] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.675Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1202gitlab # [ 109.669713] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.677Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1203gitlab # [ 109.671459] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.679Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1204gitlab # [ 109.673165] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.681Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1205gitlab # [ 109.675081] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.683Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1206gitlab # [ 109.677515] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.685Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1207gitlab # [ 109.679391] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.687Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1208gitlab # [ 109.681327] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.689Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1209gitlab # [ 109.683469] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.691Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1210gitlab # [ 109.691861] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.699Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1211gitlab # [ 109.696631] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.704Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1212gitlab # [ 109.699465] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.707Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1213gitlab # [ 109.701798] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.709Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1214gitlab # [ 109.703688] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.711Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1215gitlab # [ 109.708276] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.716Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1216gitlab # [ 109.710090] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.718Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1217gitlab # [ 109.712743] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.720Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1218gitlab # [ 109.716445] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.724Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1219gitlab # [ 109.720385] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.728Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1220gitlab # [ 109.724627] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.732Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1221gitlab # [ 109.728992] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.737Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1222gitlab # [ 109.768131] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.775Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1223gitlab # [ 109.774171] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.782Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1224gitlab # [ 109.776431] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.784Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1225gitlab # [ 109.778920] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.786Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1226gitlab # [ 109.784201] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.792Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1227gitlab # [ 109.787278] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.795Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1228gitlab # [ 109.791311] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.799Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1229gitlab # [ 109.794823] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.802Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1230gitlab # [ 109.800803] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.808Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1231gitlab # [ 109.805738] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.813Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1232gitlab # [ 109.810122] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.818Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1233gitlab # [ 109.815080] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.823Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1234gitlab # [ 109.821076] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.829Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1235gitlab # [ 109.827667] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.835Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1236gitlab # [ 109.832715] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.840Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1237gitlab # [ 109.838699] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.846Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1238gitlab # [ 109.844084] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.852Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1239gitlab # [ 109.848727] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.856Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1240gitlab # [ 109.853507] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.861Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1241gitlab # [ 109.860181] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.868Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1242gitlab # [ 109.865913] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.873Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1243gitlab # [ 109.871803] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.879Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1244gitlab # [ 109.878275] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.886Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1245gitlab # [ 109.885881] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.893Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1246gitlab # [ 109.892218] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.900Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1247gitlab # [ 109.897743] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.905Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1248gitlab # [ 109.903819] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.911Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1249gitlab # [ 109.908891] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.916Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1250gitlab # [ 109.914880] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.922Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1251gitlab # [ 109.920577] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.928Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1252gitlab # [ 109.927770] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.935Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1253gitlab # [ 109.934831] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.942Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1254gitlab # [ 109.941974] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.949Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1255gitlab # [ 109.947502] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.955Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1256gitlab # [ 109.969376] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.977Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1257gitlab # [ 109.974228] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.982Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1258gitlab # [ 109.980859] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.988Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1259gitlab # [ 109.990339] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:04.998Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1260gitlab # [ 109.999572] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.007Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1261gitlab # [ 110.009628] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.017Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1262gitlab # [ 110.019840] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.027Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1263gitlab # [ 110.027983] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.035Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1264gitlab # [ 110.035459] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.043Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1265gitlab # [ 110.045390] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.053Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1266gitlab # [ 110.075631] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:05.083Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1267gitlab # [ 118.716371] gitlab-workhorse[826]: time="2026-10-04T18:58:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1268gitlab # [ 128.718544] gitlab-workhorse[826]: time="2026-10-04T18:58:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1269gitlab # [ 132.139465] postgres[2128]: [2128] ERROR: relation "table_without_models" does not exist1270gitlab # [ 132.140001] postgres[2128]: [2128] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1271gitlab # [ 134.639840] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:29.645Z","message":"Booted Rails 7.2.3.1 application in production environment"}1272gitlab # [ 134.643149] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:29.650Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1273gitlab # [ 134.644318] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:29.650Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1274gitlab # [ 134.646256] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:29.650Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1275gitlab # [ 134.658193] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:29.666Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1276gitlab # [ 135.746317] postgres[2157]: [2157] ERROR: relation "table_without_models" does not exist1277gitlab # [ 135.747124] postgres[2157]: [2157] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1278gitlab # [ 136.199147] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:31.206Z","message":"Cleaning working queues"}1279gitlab # [ 136.204929] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:31.212Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"582f17b190698ec98e24b406","created_at":"2026-10-04T18:57:32.777Z","trace_propagation_headers":{"sentry-trace":"84adb158f7db414b878e347c8365dce0-228200cbd9e44663","baggage":"sentry-trace_id=84adb158f7db414b878e347c8365dce0,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6717fe45d96f1256862611fd77af0a4c","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:57:32.786Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-582f17b190698ec98e24b406: start","job_status":"start","queue_duration_s":58.425605,"scheduling_latency_s":58.425707}1280gitlab # [ 136.397585] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:58:31.405Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"582f17b190698ec98e24b406","created_at":"2026-10-04T18:57:32.777Z","trace_propagation_headers":{"sentry-trace":"84adb158f7db414b878e347c8365dce0-228200cbd9e44663","baggage":"sentry-trace_id=84adb158f7db414b878e347c8365dce0,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6717fe45d96f1256862611fd77af0a4c","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:57:32.786Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-582f17b190698ec98e24b406: done: 0.193072 sec","job_status":"done","queue_duration_s":58.425605,"scheduling_latency_s":58.425707,"redis_calls":31,"redis_duration_s":0.008303,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004885,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002043,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.092403923,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.193072,"completed_at":"2026-10-04T18:58:31.405Z","load_balancing_strategy":"primary","db_duration_s":0.064349,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1281gitlab # [ 138.726802] gitlab-workhorse[826]: time="2026-10-04T18:58:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1282gitlab # [ 138.784432] bundle[1632]: {"timestamp":"2026-10-04T18:58:33.791Z","pid":1632,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1283gitlab # [ 138.784746] bundle[1632]: {"timestamp":"2026-10-04T18:58:33.791Z","pid":1632,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1284gitlab # [ 138.786335] bundle[1632]: {"timestamp":"2026-10-04T18:58:33.791Z","pid":1632,"message":"! #\u003cThread:0x000073222d4dcdb8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1285gitlab # [ 138.787440] bundle[1632]: {"timestamp":"2026-10-04T18:58:33.792Z","pid":1632,"message":"! #\u003cThread:0x000073222d78d5a8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1286gitlab # [ 138.788950] bundle[1632]: {"timestamp":"2026-10-04T18:58:33.792Z","pid":1632,"message":"Use Ctrl-C to stop"}1287gitlab # [ 138.790527] bundle[1632]: {"timestamp":"2026-10-04T18:58:33.792Z","pid":1632,"message":"Sending status to systemd every 1.0 sec"}1288gitlab # [ 139.158540] bundle[1632]: {"timestamp":"2026-10-04T18:58:34.166Z","pid":1632,"message":"- Worker 0 (PID: 2220) booted in 0.32s, phase: 0"}1289gitlab # [ 139.175423] bundle[1632]: {"timestamp":"2026-10-04T18:58:34.183Z","pid":1632,"message":"- Worker 1 (PID: 2224) booted in 0.32s, phase: 0"}1290gitlab # [ 139.181614] systemd[1]: Started gitlab.service.1291gitlab # [ 139.183696] systemd[1]: Reached target Common target for all GitLab services..1292gitlab # [ 139.189243] systemd[1]: Starting setup-gitlab.service...1293gitlab # [ 139.218324] setup-gitlab-start[2257]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1294gitlab # [ 141.854651] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:58:36 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 25991295gitlab # [ 141.859773] setup-gitlab-start[2257]: + TOKEN=nixbot-test-token-of-20-chars1296gitlab # [ 141.862447] setup-gitlab-start[2257]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1297gitlab # [ 141.874860] runuser[2275]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1298gitlab # [ 142.263569] setup-gitlab-start[2276]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1299gitlab # [ 142.268616] setup-gitlab-start[2276]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1300gitlab # [ 142.273408] setup-gitlab-start[2276]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1301gitlab # [ 142.841933] setup-gitlab-start[2276]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1302gitlab # [ 142.846962] setup-gitlab-start[2276]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1303gitlab # [ 142.851778] setup-gitlab-start[2276]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1304gitlab # [ 168.116609] setup-gitlab-start[2276]: W, [2026-10-04T18:59:03.123790 #2276] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1305gitlab # [ 168.231404] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c6c7850e4909a13adcd702bd","created_at":"2026-10-04T18:59:03.171Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2acc4074f2cbff5da0be73e4c84d4c43","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.235Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c6c7850e4909a13adcd702bd: start","job_status":"start","queue_duration_s":0.002431,"scheduling_latency_s":0.002461}1306gitlab # [ 168.324549] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c6c7850e4909a13adcd702bd","created_at":"2026-10-04T18:59:03.171Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2acc4074f2cbff5da0be73e4c84d4c43","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.235Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c6c7850e4909a13adcd702bd: done: 0.093504 sec","job_status":"done","queue_duration_s":0.002431,"scheduling_latency_s":0.002461,"redis_calls":10,"redis_duration_s":0.005061,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001159,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000595327,"duration_s":0.093504,"completed_at":"2026-10-04T18:59:03.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 168.370145] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4701740d6be94903322cee35","created_at":"2026-10-04T18:59:03.249Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"808f28408a11648771763a0dba76e983","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.263Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4701740d6be94903322cee35: start","job_status":"start","queue_duration_s":0.071482,"scheduling_latency_s":0.071511}1308gitlab # [ 168.383510] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"4701740d6be94903322cee35","created_at":"2026-10-04T18:59:03.249Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"808f28408a11648771763a0dba76e983","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.263Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-4701740d6be94903322cee35: done: 0.045628 sec","job_status":"done","queue_duration_s":0.071482,"scheduling_latency_s":0.071511,"redis_calls":12,"redis_duration_s":0.003979,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000685003,"exclusive_lock_hold_duration_s":0.003048711,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.045628,"completed_at":"2026-10-04T18:59:03.380Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 168.423125] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"45ab725b84bb50e866953ac2","created_at":"2026-10-04T18:59:03.270Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6dd4d2a94c3380728f4cca0e759fe80d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T18:59:03.273Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-45ab725b84bb50e866953ac2: start","job_status":"start","queue_duration_s":0.107519,"scheduling_latency_s":0.107534}1310gitlab # [ 168.435924] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.394Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1311gitlab # [ 168.439624] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"45ab725b84bb50e866953ac2","created_at":"2026-10-04T18:59:03.270Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6dd4d2a94c3380728f4cca0e759fe80d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T18:59:03.273Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-45ab725b84bb50e866953ac2: done: 0.037031 sec","job_status":"done","queue_duration_s":0.107519,"scheduling_latency_s":0.107534,"redis_calls":10,"redis_duration_s":0.001311,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214274,"duration_s":0.037031,"completed_at":"2026-10-04T18:59:03.418Z","load_balancing_strategy":"primary","db_duration_s":0.001069,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 168.478287] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3238b5b6598c4eac7bd0640d","created_at":"2026-10-04T18:59:03.277Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbc8987eb86f4aacf7bd1460603a988f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T18:59:03.280Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3238b5b6598c4eac7bd0640d: start","job_status":"start","queue_duration_s":0.139032,"scheduling_latency_s":0.139048}1313gitlab # [ 168.478659] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3238b5b6598c4eac7bd0640d","created_at":"2026-10-04T18:59:03.277Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbc8987eb86f4aacf7bd1460603a988f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T18:59:03.280Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3238b5b6598c4eac7bd0640d: done: 0.029662 sec","job_status":"done","queue_duration_s":0.139032,"scheduling_latency_s":0.139048,"redis_calls":10,"redis_duration_s":0.001287,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207289,"duration_s":0.029662,"completed_at":"2026-10-04T18:59:03.449Z","load_balancing_strategy":"primary","db_duration_s":0.00906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 168.486876] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a8ec88bf59fd2cfc83f31622","created_at":"2026-10-04T18:59:03.284Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f527ee2b73a1124457379a4a8c117dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.288Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a8ec88bf59fd2cfc83f31622: start","job_status":"start","queue_duration_s":0.162378,"scheduling_latency_s":0.162392}1315gitlab # [ 168.526661] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.478Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1316gitlab # [ 168.527489] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.485Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a8ec88bf59fd2cfc83f31622","created_at":"2026-10-04T18:59:03.284Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f527ee2b73a1124457379a4a8c117dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.288Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a8ec88bf59fd2cfc83f31622: done: 0.035093 sec","job_status":"done","queue_duration_s":0.162378,"scheduling_latency_s":0.162392,"redis_calls":10,"redis_duration_s":0.001252,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000206172,"duration_s":0.035093,"completed_at":"2026-10-04T18:59:03.485Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 168.542517] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a8ea6cc4004e60c1a3a19197","created_at":"2026-10-04T18:59:03.310Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f18caef1458959d6dfd4c4975f4f9dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.330Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a8ea6cc4004e60c1a3a19197: start","job_status":"start","queue_duration_s":0.171731,"scheduling_latency_s":0.171751}1318gitlab # [ 168.583808] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.512Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1319gitlab # [ 168.584798] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a8ea6cc4004e60c1a3a19197","created_at":"2026-10-04T18:59:03.310Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f18caef1458959d6dfd4c4975f4f9dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.330Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a8ea6cc4004e60c1a3a19197: done: 0.015589 sec","job_status":"done","queue_duration_s":0.171731,"scheduling_latency_s":0.171751,"redis_calls":10,"redis_duration_s":0.001598,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000216787,"duration_s":0.015589,"completed_at":"2026-10-04T18:59:03.517Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 168.597071] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"68f78d153ca841314466cb8b","created_at":"2026-10-04T18:59:03.343Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.384Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-68f78d153ca841314466cb8b: start","job_status":"start","queue_duration_s":0.133582,"scheduling_latency_s":0.133595}1321gitlab # [ 168.610798] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.552Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"68f78d153ca841314466cb8b","created_at":"2026-10-04T18:59:03.343Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.384Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-68f78d153ca841314466cb8b: done: 0.033959 sec","job_status":"done","queue_duration_s":0.133582,"scheduling_latency_s":0.133595,"redis_calls":19,"redis_duration_s":0.008236,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00051,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000253,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006721,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210641,"duration_s":0.033959,"completed_at":"2026-10-04T18:59:03.552Z","load_balancing_strategy":"primary","db_duration_s":0.00248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 168.655809] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.553Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6a9751d4e48978c086d060c9","created_at":"2026-10-04T18:59:03.412Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.438Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6a9751d4e48978c086d060c9: start","job_status":"start","queue_duration_s":0.115272,"scheduling_latency_s":0.115287}1323gitlab # [ 168.663428] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.619Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6a9751d4e48978c086d060c9","created_at":"2026-10-04T18:59:03.412Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.438Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6a9751d4e48978c086d060c9: done: 0.066426 sec","job_status":"done","queue_duration_s":0.115272,"scheduling_latency_s":0.115287,"redis_calls":71,"redis_duration_s":0.009996999999999999,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001267,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001004,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.007336,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222933,"duration_s":0.066426,"completed_at":"2026-10-04T18:59:03.619Z","load_balancing_strategy":"primary","db_duration_s":0.002107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 168.715151] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"de262167f184281e6496f204","created_at":"2026-10-04T18:59:03.463Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19b90d2b7dcc60ff160747778dd9a259","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.477Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-de262167f184281e6496f204: start","job_status":"start","queue_duration_s":0.143258,"scheduling_latency_s":0.143273}1325gitlab # [ 168.716324] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"de262167f184281e6496f204","created_at":"2026-10-04T18:59:03.463Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19b90d2b7dcc60ff160747778dd9a259","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.477Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-de262167f184281e6496f204: done: 0.062909 sec","job_status":"done","queue_duration_s":0.143258,"scheduling_latency_s":0.143273,"redis_calls":12,"redis_duration_s":0.005052,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00040061,"exclusive_lock_hold_duration_s":0.031017628999999998,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.062909,"completed_at":"2026-10-04T18:59:03.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 168.758851] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.684Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eabe669ddcb023468f560ea9","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eabe669ddcb023468f560ea9: start","job_status":"start","queue_duration_s":0.140371,"scheduling_latency_s":0.140387}1327gitlab # [ 168.801458] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.696Z","message":"No existing merge request to be cleaned up."}1328gitlab # [ 168.802332] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.705Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eabe669ddcb023468f560ea9","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eabe669ddcb023468f560ea9: done: 0.020864 sec","job_status":"done","queue_duration_s":0.140371,"scheduling_latency_s":0.140387,"redis_calls":13,"redis_duration_s":0.001822,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203936,"duration_s":0.020864,"completed_at":"2026-10-04T18:59:03.705Z","load_balancing_strategy":"primary","db_duration_s":0.004675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 168.803942] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.706Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"825abfa9174dc397b0f793db","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-825abfa9174dc397b0f793db: start","job_status":"start","queue_duration_s":0.16209,"scheduling_latency_s":0.162103}1330gitlab # [ 168.845959] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.713Z","message":"No existing merge request to be cleaned up."}1331gitlab # [ 168.846768] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.717Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"825abfa9174dc397b0f793db","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-825abfa9174dc397b0f793db: done: 0.011472 sec","job_status":"done","queue_duration_s":0.16209,"scheduling_latency_s":0.162103,"redis_calls":10,"redis_duration_s":0.001253,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011472,"completed_at":"2026-10-04T18:59:03.717Z","load_balancing_strategy":"primary","db_duration_s":0.001723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 168.852423] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.718Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"790b42c8c7693f342f4bb2b0","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-790b42c8c7693f342f4bb2b0: start","job_status":"start","queue_duration_s":0.174411,"scheduling_latency_s":0.174424}1333gitlab # [ 168.894179] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.725Z","message":"No existing merge request to be cleaned up."}1334gitlab # [ 168.894915] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.730Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"790b42c8c7693f342f4bb2b0","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-790b42c8c7693f342f4bb2b0: done: 0.01159 sec","job_status":"done","queue_duration_s":0.174411,"scheduling_latency_s":0.174424,"redis_calls":10,"redis_duration_s":0.001361,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01159,"completed_at":"2026-10-04T18:59:03.730Z","load_balancing_strategy":"primary","db_duration_s":0.001766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 168.899975] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.731Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"98c9ee31fc8957c90fa11474","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-98c9ee31fc8957c90fa11474: start","job_status":"start","queue_duration_s":0.186922,"scheduling_latency_s":0.186936}1336gitlab # [ 168.939141] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.737Z","message":"No existing merge request to be cleaned up."}1337gitlab # [ 168.939929] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.742Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"98c9ee31fc8957c90fa11474","created_at":"2026-10-04T18:59:03.529Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-5bc9bc25c9c4473f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3157e1e56762bca0d890fa57fac4d95c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.544Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-98c9ee31fc8957c90fa11474: done: 0.01138 sec","job_status":"done","queue_duration_s":0.186922,"scheduling_latency_s":0.186936,"redis_calls":10,"redis_duration_s":0.001251,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01138,"completed_at":"2026-10-04T18:59:03.742Z","load_balancing_strategy":"primary","db_duration_s":0.001813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 168.947265] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"143734ff1ad7dc50694ca159","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-143734ff1ad7dc50694ca159: start","job_status":"start","queue_duration_s":0.13035,"scheduling_latency_s":0.130363}1339gitlab # [ 168.958905] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"143734ff1ad7dc50694ca159","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-143734ff1ad7dc50694ca159: done: 0.013639 sec","job_status":"done","queue_duration_s":0.13035,"scheduling_latency_s":0.130363,"redis_calls":16,"redis_duration_s":0.002028,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013639,"completed_at":"2026-10-04T18:59:03.757Z","load_balancing_strategy":"primary","db_duration_s":0.000442,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 169.009681] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ab7f62df8ccdd20d63e8552","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ab7f62df8ccdd20d63e8552: start","job_status":"start","queue_duration_s":0.144823,"scheduling_latency_s":0.144836}1341gitlab # [ 169.010559] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ab7f62df8ccdd20d63e8552","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ab7f62df8ccdd20d63e8552: done: 0.009153 sec","job_status":"done","queue_duration_s":0.144823,"scheduling_latency_s":0.144836,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009153,"completed_at":"2026-10-04T18:59:03.767Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 169.050753] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26eca869ad102025a47cb205","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26eca869ad102025a47cb205: start","job_status":"start","queue_duration_s":0.15477,"scheduling_latency_s":0.154783}1343gitlab # [ 169.054767] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26eca869ad102025a47cb205","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26eca869ad102025a47cb205: done: 0.009 sec","job_status":"done","queue_duration_s":0.15477,"scheduling_latency_s":0.154783,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009,"completed_at":"2026-10-04T18:59:03.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 169.098718] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d148a81031b4741afa1c124","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d148a81031b4741afa1c124: start","job_status":"start","queue_duration_s":0.164668,"scheduling_latency_s":0.164681}1345gitlab # [ 169.100908] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d148a81031b4741afa1c124","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d148a81031b4741afa1c124: done: 0.009079 sec","job_status":"done","queue_duration_s":0.164668,"scheduling_latency_s":0.164681,"redis_calls":13,"redis_duration_s":0.001545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009079,"completed_at":"2026-10-04T18:59:03.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 169.162800] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cd79a6b651ddc9d9f054aba","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cd79a6b651ddc9d9f054aba: start","job_status":"start","queue_duration_s":0.174582,"scheduling_latency_s":0.174595}1347gitlab # [ 169.163681] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cd79a6b651ddc9d9f054aba","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cd79a6b651ddc9d9f054aba: done: 0.008948 sec","job_status":"done","queue_duration_s":0.174582,"scheduling_latency_s":0.174595,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008948,"completed_at":"2026-10-04T18:59:03.796Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 169.222074] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eed38c298226d2d7dac0368","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eed38c298226d2d7dac0368: start","job_status":"start","queue_duration_s":0.184461,"scheduling_latency_s":0.184474}1349gitlab # [ 169.222871] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eed38c298226d2d7dac0368","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eed38c298226d2d7dac0368: done: 0.00929 sec","job_status":"done","queue_duration_s":0.184461,"scheduling_latency_s":0.184474,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00929,"completed_at":"2026-10-04T18:59:03.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 169.266949] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69451cbba2c2030faed464f","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69451cbba2c2030faed464f: start","job_status":"start","queue_duration_s":0.194577,"scheduling_latency_s":0.19459}1351gitlab # [ 169.267671] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69451cbba2c2030faed464f","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69451cbba2c2030faed464f: done: 0.009049 sec","job_status":"done","queue_duration_s":0.194577,"scheduling_latency_s":0.19459,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009049,"completed_at":"2026-10-04T18:59:03.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 169.282229] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13228aec42c3b58825372c22","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13228aec42c3b58825372c22: start","job_status":"start","queue_duration_s":0.204582,"scheduling_latency_s":0.204597}1353gitlab # [ 169.294193] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13228aec42c3b58825372c22","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13228aec42c3b58825372c22: done: 0.009086 sec","job_status":"done","queue_duration_s":0.204582,"scheduling_latency_s":0.204597,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009086,"completed_at":"2026-10-04T18:59:03.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 169.336217] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ea5e5ed05699c445ba23deb","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ea5e5ed05699c445ba23deb: start","job_status":"start","queue_duration_s":0.214597,"scheduling_latency_s":0.21461}1355gitlab # [ 169.342509] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ea5e5ed05699c445ba23deb","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ea5e5ed05699c445ba23deb: done: 0.009095 sec","job_status":"done","queue_duration_s":0.214597,"scheduling_latency_s":0.21461,"redis_calls":13,"redis_duration_s":0.001501,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009095,"completed_at":"2026-10-04T18:59:03.836Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 169.384768] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea72b5dfba9ae3b78f18bdc3","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea72b5dfba9ae3b78f18bdc3: start","job_status":"start","queue_duration_s":0.224573,"scheduling_latency_s":0.224587}1357gitlab # [ 169.390870] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea72b5dfba9ae3b78f18bdc3","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea72b5dfba9ae3b78f18bdc3: done: 0.009144 sec","job_status":"done","queue_duration_s":0.224573,"scheduling_latency_s":0.224587,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009144,"completed_at":"2026-10-04T18:59:03.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 169.438791] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d54ff6591490eeb2e927b1a5","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d54ff6591490eeb2e927b1a5: start","job_status":"start","queue_duration_s":0.234553,"scheduling_latency_s":0.234566}1359gitlab # [ 169.439730] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d54ff6591490eeb2e927b1a5","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d54ff6591490eeb2e927b1a5: done: 0.009045 sec","job_status":"done","queue_duration_s":0.234553,"scheduling_latency_s":0.234566,"redis_calls":13,"redis_duration_s":0.0016480000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009045,"completed_at":"2026-10-04T18:59:03.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 169.489531] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7b0305d40196bf5926673b","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7b0305d40196bf5926673b: start","job_status":"start","queue_duration_s":0.24443,"scheduling_latency_s":0.244443}1361gitlab # [ 169.490287] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7b0305d40196bf5926673b","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7b0305d40196bf5926673b: done: 0.009437 sec","job_status":"done","queue_duration_s":0.24443,"scheduling_latency_s":0.244443,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009437,"completed_at":"2026-10-04T18:59:03.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 169.538807] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd2cbf62d984e3a6470c7fa0","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd2cbf62d984e3a6470c7fa0: start","job_status":"start","queue_duration_s":0.254695,"scheduling_latency_s":0.254708}1363gitlab # [ 169.539500] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd2cbf62d984e3a6470c7fa0","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd2cbf62d984e3a6470c7fa0: done: 0.008927 sec","job_status":"done","queue_duration_s":0.254695,"scheduling_latency_s":0.254708,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008927,"completed_at":"2026-10-04T18:59:03.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 169.605984] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"813529de961e9c6d9823e117","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-813529de961e9c6d9823e117: start","job_status":"start","queue_duration_s":0.264571,"scheduling_latency_s":0.264583}1365gitlab # [ 169.606734] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"813529de961e9c6d9823e117","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-813529de961e9c6d9823e117: done: 0.00904 sec","job_status":"done","queue_duration_s":0.264571,"scheduling_latency_s":0.264583,"redis_calls":13,"redis_duration_s":0.0014849999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00904,"completed_at":"2026-10-04T18:59:03.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 169.647922] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f9b07190f26763af2b7d19c","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f9b07190f26763af2b7d19c: start","job_status":"start","queue_duration_s":0.274449,"scheduling_latency_s":0.274461}1367gitlab # [ 169.648732] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f9b07190f26763af2b7d19c","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f9b07190f26763af2b7d19c: done: 0.009003 sec","job_status":"done","queue_duration_s":0.274449,"scheduling_latency_s":0.274461,"redis_calls":13,"redis_duration_s":0.001647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009003,"completed_at":"2026-10-04T18:59:03.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 169.690164] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c0c3718fa38cc2d524b5d5","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c0c3718fa38cc2d524b5d5: start","job_status":"start","queue_duration_s":0.284282,"scheduling_latency_s":0.284295}1369gitlab # [ 169.690890] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6c0c3718fa38cc2d524b5d5","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6c0c3718fa38cc2d524b5d5: done: 0.00951 sec","job_status":"done","queue_duration_s":0.284282,"scheduling_latency_s":0.284295,"redis_calls":13,"redis_duration_s":0.001735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00951,"completed_at":"2026-10-04T18:59:03.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 169.737548] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b8f0277215fa4fb06166ede","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b8f0277215fa4fb06166ede: start","job_status":"start","queue_duration_s":0.294653,"scheduling_latency_s":0.294666}1371gitlab # [ 169.738186] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b8f0277215fa4fb06166ede","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b8f0277215fa4fb06166ede: done: 0.009135 sec","job_status":"done","queue_duration_s":0.294653,"scheduling_latency_s":0.294666,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009135,"completed_at":"2026-10-04T18:59:03.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 169.779076] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee92778e3fe4207586595fcc","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee92778e3fe4207586595fcc: start","job_status":"start","queue_duration_s":0.304738,"scheduling_latency_s":0.304752}1373gitlab # [ 169.779861] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee92778e3fe4207586595fcc","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee92778e3fe4207586595fcc: done: 0.009066 sec","job_status":"done","queue_duration_s":0.304738,"scheduling_latency_s":0.304752,"redis_calls":13,"redis_duration_s":0.0014939999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009066,"completed_at":"2026-10-04T18:59:03.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 169.834922] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f469c6a68d004bc5402833b6","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f469c6a68d004bc5402833b6: start","job_status":"start","queue_duration_s":0.314649,"scheduling_latency_s":0.314663}1375gitlab # [ 169.835748] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f469c6a68d004bc5402833b6","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f469c6a68d004bc5402833b6: done: 0.009005 sec","job_status":"done","queue_duration_s":0.314649,"scheduling_latency_s":0.314663,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009005,"completed_at":"2026-10-04T18:59:03.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 169.887552] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6917ad599c9f2104d6b2c62","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6917ad599c9f2104d6b2c62: start","job_status":"start","queue_duration_s":0.324487,"scheduling_latency_s":0.324501}1377gitlab # [ 169.888221] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6917ad599c9f2104d6b2c62","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6917ad599c9f2104d6b2c62: done: 0.009043 sec","job_status":"done","queue_duration_s":0.324487,"scheduling_latency_s":0.324501,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009043,"completed_at":"2026-10-04T18:59:03.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 169.931092] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b94f4a10ba4a5bf653015e","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b94f4a10ba4a5bf653015e: start","job_status":"start","queue_duration_s":0.334444,"scheduling_latency_s":0.334458}1379gitlab # [ 169.931918] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44b94f4a10ba4a5bf653015e","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44b94f4a10ba4a5bf653015e: done: 0.009124 sec","job_status":"done","queue_duration_s":0.334444,"scheduling_latency_s":0.334458,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009124,"completed_at":"2026-10-04T18:59:03.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 169.992447] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2594212107480c2fac264797","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2594212107480c2fac264797: start","job_status":"start","queue_duration_s":0.34439,"scheduling_latency_s":0.344403}1381gitlab # [ 169.993231] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2594212107480c2fac264797","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2594212107480c2fac264797: done: 0.035657 sec","job_status":"done","queue_duration_s":0.34439,"scheduling_latency_s":0.344403,"redis_calls":13,"redis_duration_s":0.014830999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.012247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035657,"completed_at":"2026-10-04T18:59:03.993Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 170.003782] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:03.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7b7258a7d45abc4fccd885b","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7b7258a7d45abc4fccd885b: start","job_status":"start","queue_duration_s":0.381523,"scheduling_latency_s":0.381548}1383gitlab # [ 170.045542] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7b7258a7d45abc4fccd885b","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7b7258a7d45abc4fccd885b: done: 0.023908 sec","job_status":"done","queue_duration_s":0.381523,"scheduling_latency_s":0.381548,"redis_calls":13,"redis_duration_s":0.010950999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023908,"completed_at":"2026-10-04T18:59:04.018Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 170.049399] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33ddbaa4a8f97b6c671ffaca","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33ddbaa4a8f97b6c671ffaca: start","job_status":"start","queue_duration_s":0.407333,"scheduling_latency_s":0.40735}1385gitlab # [ 170.094786] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33ddbaa4a8f97b6c671ffaca","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33ddbaa4a8f97b6c671ffaca: done: 0.030149 sec","job_status":"done","queue_duration_s":0.407333,"scheduling_latency_s":0.40735,"redis_calls":13,"redis_duration_s":0.020447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.015254,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030149,"completed_at":"2026-10-04T18:59:04.050Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 170.095919] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7fdeac7fd1f80482546def6","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7fdeac7fd1f80482546def6: start","job_status":"start","queue_duration_s":0.440842,"scheduling_latency_s":0.440885}1387gitlab # [ 170.140799] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7fdeac7fd1f80482546def6","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7fdeac7fd1f80482546def6: done: 0.037167 sec","job_status":"done","queue_duration_s":0.440842,"scheduling_latency_s":0.440885,"redis_calls":13,"redis_duration_s":0.016773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00165,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037167,"completed_at":"2026-10-04T18:59:04.091Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 170.141505] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0889a6ad457c5243936438c0","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0889a6ad457c5243936438c0: start","job_status":"start","queue_duration_s":0.480125,"scheduling_latency_s":0.480145}1389gitlab # [ 170.151955] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0889a6ad457c5243936438c0","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0889a6ad457c5243936438c0: done: 0.025314 sec","job_status":"done","queue_duration_s":0.480125,"scheduling_latency_s":0.480145,"redis_calls":13,"redis_duration_s":0.014866999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.006004,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025314,"completed_at":"2026-10-04T18:59:04.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 170.193098] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bce0b85012f59a7f59f5232","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bce0b85012f59a7f59f5232: start","job_status":"start","queue_duration_s":0.508209,"scheduling_latency_s":0.508224}1391gitlab # [ 170.197983] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bce0b85012f59a7f59f5232","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bce0b85012f59a7f59f5232: done: 0.017024 sec","job_status":"done","queue_duration_s":0.508209,"scheduling_latency_s":0.508224,"redis_calls":13,"redis_duration_s":0.004673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017024,"completed_at":"2026-10-04T18:59:04.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 170.251535] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9308f330462279af47d079fc","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9308f330462279af47d079fc: start","job_status":"start","queue_duration_s":0.527641,"scheduling_latency_s":0.527655}1393gitlab # [ 170.252278] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9308f330462279af47d079fc","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9308f330462279af47d079fc: done: 0.016589 sec","job_status":"done","queue_duration_s":0.527641,"scheduling_latency_s":0.527655,"redis_calls":13,"redis_duration_s":0.007305,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001283,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016589,"completed_at":"2026-10-04T18:59:04.157Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 170.296463] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ab84a2a6761ea1450a1a4f6","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ab84a2a6761ea1450a1a4f6: start","job_status":"start","queue_duration_s":0.547211,"scheduling_latency_s":0.547237}1395gitlab # [ 170.297276] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ab84a2a6761ea1450a1a4f6","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ab84a2a6761ea1450a1a4f6: done: 0.011348 sec","job_status":"done","queue_duration_s":0.547211,"scheduling_latency_s":0.547237,"redis_calls":13,"redis_duration_s":0.001671,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011348,"completed_at":"2026-10-04T18:59:04.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 170.336633] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c82c18b973f5102d3b80cdf5","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c82c18b973f5102d3b80cdf5: start","job_status":"start","queue_duration_s":0.559514,"scheduling_latency_s":0.559529}1397gitlab # [ 170.337557] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c82c18b973f5102d3b80cdf5","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c82c18b973f5102d3b80cdf5: done: 0.010202 sec","job_status":"done","queue_duration_s":0.559514,"scheduling_latency_s":0.559529,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010202,"completed_at":"2026-10-04T18:59:04.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 170.385699] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d960c5ba6d594f17b49c2957","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d960c5ba6d594f17b49c2957: start","job_status":"start","queue_duration_s":0.570576,"scheduling_latency_s":0.57059}1399gitlab # [ 170.386432] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d960c5ba6d594f17b49c2957","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d960c5ba6d594f17b49c2957: done: 0.00946 sec","job_status":"done","queue_duration_s":0.570576,"scheduling_latency_s":0.57059,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00946,"completed_at":"2026-10-04T18:59:04.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 170.427072] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b701b40ab4ee647f35c24349","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b701b40ab4ee647f35c24349: start","job_status":"start","queue_duration_s":0.580882,"scheduling_latency_s":0.580896}1401gitlab # [ 170.427984] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b701b40ab4ee647f35c24349","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b701b40ab4ee647f35c24349: done: 0.009006 sec","job_status":"done","queue_duration_s":0.580882,"scheduling_latency_s":0.580896,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009006,"completed_at":"2026-10-04T18:59:04.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 170.489773] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af1f2bfa393994fbfda4ed6f","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af1f2bfa393994fbfda4ed6f: start","job_status":"start","queue_duration_s":0.590695,"scheduling_latency_s":0.590709}1403gitlab # [ 170.490698] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af1f2bfa393994fbfda4ed6f","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af1f2bfa393994fbfda4ed6f: done: 0.009165 sec","job_status":"done","queue_duration_s":0.590695,"scheduling_latency_s":0.590709,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009165,"completed_at":"2026-10-04T18:59:04.213Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 170.530995] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624a16c16d67702ff59a41eb","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624a16c16d67702ff59a41eb: start","job_status":"start","queue_duration_s":0.600702,"scheduling_latency_s":0.600714}1405gitlab # [ 170.531539] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624a16c16d67702ff59a41eb","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624a16c16d67702ff59a41eb: done: 0.008903 sec","job_status":"done","queue_duration_s":0.600702,"scheduling_latency_s":0.600714,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008903,"completed_at":"2026-10-04T18:59:04.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 170.577107] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd8b1a840c1e051eba94e947","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd8b1a840c1e051eba94e947: start","job_status":"start","queue_duration_s":0.610428,"scheduling_latency_s":0.61044}1407gitlab # [ 170.577905] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd8b1a840c1e051eba94e947","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd8b1a840c1e051eba94e947: done: 0.009603 sec","job_status":"done","queue_duration_s":0.610428,"scheduling_latency_s":0.61044,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009603,"completed_at":"2026-10-04T18:59:04.233Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 170.628821] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00a40a36a23bf4f114a5e789","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00a40a36a23bf4f114a5e789: start","job_status":"start","queue_duration_s":0.620884,"scheduling_latency_s":0.620898}1409gitlab # [ 170.629684] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00a40a36a23bf4f114a5e789","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00a40a36a23bf4f114a5e789: done: 0.011696 sec","job_status":"done","queue_duration_s":0.620884,"scheduling_latency_s":0.620898,"redis_calls":13,"redis_duration_s":0.001739,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011696,"completed_at":"2026-10-04T18:59:04.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 170.679287] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e83e48e99a3c06e042ff3ee","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e83e48e99a3c06e042ff3ee: start","job_status":"start","queue_duration_s":0.633668,"scheduling_latency_s":0.633691}1411gitlab # [ 170.680075] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e83e48e99a3c06e042ff3ee","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e83e48e99a3c06e042ff3ee: done: 0.009933 sec","job_status":"done","queue_duration_s":0.633668,"scheduling_latency_s":0.633691,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009933,"completed_at":"2026-10-04T18:59:04.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 170.692703] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ac50f8c0301cd42394ed89a","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ac50f8c0301cd42394ed89a: start","job_status":"start","queue_duration_s":0.644406,"scheduling_latency_s":0.644419}1413gitlab # [ 170.736279] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ac50f8c0301cd42394ed89a","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ac50f8c0301cd42394ed89a: done: 0.008932 sec","job_status":"done","queue_duration_s":0.644406,"scheduling_latency_s":0.644419,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008932,"completed_at":"2026-10-04T18:59:04.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 170.738903] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c811745ec42c92c9e936d048","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c811745ec42c92c9e936d048: start","job_status":"start","queue_duration_s":0.654164,"scheduling_latency_s":0.654178}1415gitlab # [ 170.750474] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c811745ec42c92c9e936d048","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c811745ec42c92c9e936d048: done: 0.010314 sec","job_status":"done","queue_duration_s":0.654164,"scheduling_latency_s":0.654178,"redis_calls":13,"redis_duration_s":0.001729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010314,"completed_at":"2026-10-04T18:59:04.277Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 170.790058] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d143386269fd91311cd3939","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d143386269fd91311cd3939: start","job_status":"start","queue_duration_s":0.665497,"scheduling_latency_s":0.665516}1417gitlab # [ 170.796096] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d143386269fd91311cd3939","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d143386269fd91311cd3939: done: 0.009725 sec","job_status":"done","queue_duration_s":0.665497,"scheduling_latency_s":0.665516,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009725,"completed_at":"2026-10-04T18:59:04.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 170.842735] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dba5f91e3723e374e5ead7e","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dba5f91e3723e374e5ead7e: start","job_status":"start","queue_duration_s":0.676111,"scheduling_latency_s":0.676125}1419gitlab # [ 170.843694] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dba5f91e3723e374e5ead7e","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dba5f91e3723e374e5ead7e: done: 0.009136 sec","job_status":"done","queue_duration_s":0.676111,"scheduling_latency_s":0.676125,"redis_calls":13,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009136,"completed_at":"2026-10-04T18:59:04.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 170.877999] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a998489054b62082ee12bb9a","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a998489054b62082ee12bb9a: start","job_status":"start","queue_duration_s":0.686124,"scheduling_latency_s":0.686137}1421gitlab # [ 170.924216] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a998489054b62082ee12bb9a","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a998489054b62082ee12bb9a: done: 0.009432 sec","job_status":"done","queue_duration_s":0.686124,"scheduling_latency_s":0.686137,"redis_calls":13,"redis_duration_s":0.001941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001409,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009432,"completed_at":"2026-10-04T18:59:04.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 170.925201] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a276edeb018c28b153137004","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a276edeb018c28b153137004: start","job_status":"start","queue_duration_s":0.696395,"scheduling_latency_s":0.696408}1423gitlab # [ 170.968792] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a276edeb018c28b153137004","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a276edeb018c28b153137004: done: 0.009011 sec","job_status":"done","queue_duration_s":0.696395,"scheduling_latency_s":0.696408,"redis_calls":13,"redis_duration_s":0.0015110000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009011,"completed_at":"2026-10-04T18:59:04.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 170.969955] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16556f303d8fc580236d61dd","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16556f303d8fc580236d61dd: start","job_status":"start","queue_duration_s":0.706253,"scheduling_latency_s":0.706267}1425gitlab # [ 171.008529] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16556f303d8fc580236d61dd","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16556f303d8fc580236d61dd: done: 0.008927 sec","job_status":"done","queue_duration_s":0.706253,"scheduling_latency_s":0.706267,"redis_calls":13,"redis_duration_s":0.001512,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008927,"completed_at":"2026-10-04T18:59:04.328Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 171.015848] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9eb92382633101d237e0365","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9eb92382633101d237e0365: start","job_status":"start","queue_duration_s":0.715991,"scheduling_latency_s":0.716005}1427gitlab # [ 171.055288] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9eb92382633101d237e0365","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9eb92382633101d237e0365: done: 0.00893 sec","job_status":"done","queue_duration_s":0.715991,"scheduling_latency_s":0.716005,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00893,"completed_at":"2026-10-04T18:59:04.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 171.099755] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4de04933369e6151e848b61","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4de04933369e6151e848b61: start","job_status":"start","queue_duration_s":0.725736,"scheduling_latency_s":0.72575}1429gitlab # [ 171.100570] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4de04933369e6151e848b61","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4de04933369e6151e848b61: done: 0.009187 sec","job_status":"done","queue_duration_s":0.725736,"scheduling_latency_s":0.72575,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009187,"completed_at":"2026-10-04T18:59:04.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 171.107812] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb0965848d0a0517336315cc","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb0965848d0a0517336315cc: start","job_status":"start","queue_duration_s":0.735782,"scheduling_latency_s":0.735798}1431gitlab # [ 171.149512] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb0965848d0a0517336315cc","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb0965848d0a0517336315cc: done: 0.009547 sec","job_status":"done","queue_duration_s":0.735782,"scheduling_latency_s":0.735798,"redis_calls":13,"redis_duration_s":0.0017760000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009547,"completed_at":"2026-10-04T18:59:04.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 171.153643] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"286bddf0baf86f31ee30a7c9","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-286bddf0baf86f31ee30a7c9: start","job_status":"start","queue_duration_s":0.746203,"scheduling_latency_s":0.746217}1433gitlab # [ 171.201711] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"286bddf0baf86f31ee30a7c9","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-286bddf0baf86f31ee30a7c9: done: 0.009141 sec","job_status":"done","queue_duration_s":0.746203,"scheduling_latency_s":0.746217,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009141,"completed_at":"2026-10-04T18:59:04.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 171.202773] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4298e8037ebdfb18ff4ce315","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4298e8037ebdfb18ff4ce315: start","job_status":"start","queue_duration_s":0.756235,"scheduling_latency_s":0.756249}1435gitlab # [ 171.247625] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4298e8037ebdfb18ff4ce315","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4298e8037ebdfb18ff4ce315: done: 0.009172 sec","job_status":"done","queue_duration_s":0.756235,"scheduling_latency_s":0.756249,"redis_calls":13,"redis_duration_s":0.0016480000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009172,"completed_at":"2026-10-04T18:59:04.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 171.248629] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17324ab47ecf6b8345c0c52f","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17324ab47ecf6b8345c0c52f: start","job_status":"start","queue_duration_s":0.76629,"scheduling_latency_s":0.766304}1437gitlab # [ 171.291871] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17324ab47ecf6b8345c0c52f","created_at":"2026-10-04T18:59:03.566Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9aa0f82d3a2c4975","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"9db692cde5ca825b81e3878d4ca4537e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.613Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17324ab47ecf6b8345c0c52f: done: 0.009174 sec","job_status":"done","queue_duration_s":0.76629,"scheduling_latency_s":0.766304,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009174,"completed_at":"2026-10-04T18:59:04.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 171.293042] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e9528127605192498f757ee6","created_at":"2026-10-04T18:59:03.499Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2556ce0ba41cfa6b1693234d40820f97","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.971Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e9528127605192498f757ee6: start","job_status":"start","queue_duration_s":0.418043,"scheduling_latency_s":0.418056}1439gitlab # [ 171.340470] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e9528127605192498f757ee6","created_at":"2026-10-04T18:59:03.499Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2556ce0ba41cfa6b1693234d40820f97","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.971Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e9528127605192498f757ee6: done: 0.024398 sec","job_status":"done","queue_duration_s":0.418043,"scheduling_latency_s":0.418056,"redis_calls":16,"redis_duration_s":0.002271,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001274,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227962,"duration_s":0.024398,"completed_at":"2026-10-04T18:59:04.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 171.341678] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"42de505158e3c86605cd5bb6","created_at":"2026-10-04T18:59:03.985Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e6b0fb4fde202e143c8efbc423ed07d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.992Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-42de505158e3c86605cd5bb6: start","job_status":"start","queue_duration_s":0.42241,"scheduling_latency_s":0.422424}1441gitlab # [ 171.389169] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.421Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1442gitlab # [ 171.390034] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.425Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"42de505158e3c86605cd5bb6","created_at":"2026-10-04T18:59:03.985Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e6b0fb4fde202e143c8efbc423ed07d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:03.992Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-42de505158e3c86605cd5bb6: done: 0.010677 sec","job_status":"done","queue_duration_s":0.42241,"scheduling_latency_s":0.422424,"redis_calls":10,"redis_duration_s":0.001375,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239975,"duration_s":0.010677,"completed_at":"2026-10-04T18:59:04.425Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 171.394930] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8cd396fe9778458d064c1479","created_at":"2026-10-04T18:59:03.999Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"239cd0c6d5c1853896b850f22d572d15","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.005Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8cd396fe9778458d064c1479: start","job_status":"start","queue_duration_s":0.421086,"scheduling_latency_s":0.421099}1444gitlab # [ 171.407612] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8cd396fe9778458d064c1479","created_at":"2026-10-04T18:59:03.999Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"239cd0c6d5c1853896b850f22d572d15","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.005Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8cd396fe9778458d064c1479: done: 0.022196 sec","job_status":"done","queue_duration_s":0.421086,"scheduling_latency_s":0.421099,"redis_calls":10,"redis_duration_s":0.001495,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243048,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.022196,"completed_at":"2026-10-04T18:59:04.448Z","load_balancing_strategy":"primary","db_duration_s":0.00912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 171.471143] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"788eb79d506bac6036f361ee","created_at":"2026-10-04T18:59:04.012Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3da7c1606cb625111cf047aaa1c8f362","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.019Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-788eb79d506bac6036f361ee: start","job_status":"start","queue_duration_s":0.43007,"scheduling_latency_s":0.430084}1446gitlab # [ 171.471898] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"788eb79d506bac6036f361ee","created_at":"2026-10-04T18:59:04.012Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3da7c1606cb625111cf047aaa1c8f362","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.019Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-788eb79d506bac6036f361ee: done: 0.014246 sec","job_status":"done","queue_duration_s":0.43007,"scheduling_latency_s":0.430084,"redis_calls":12,"redis_duration_s":0.001533,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000561,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.014246,"completed_at":"2026-10-04T18:59:04.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002157,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 171.514084] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"dfb0f19c2a634c46e33e64b9","created_at":"2026-10-04T18:59:04.037Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80f81a3d79e279a58fab7d0958da27ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.044Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-dfb0f19c2a634c46e33e64b9: start","job_status":"start","queue_duration_s":0.420549,"scheduling_latency_s":0.420561}1448gitlab # [ 171.514820] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"dfb0f19c2a634c46e33e64b9","created_at":"2026-10-04T18:59:04.037Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80f81a3d79e279a58fab7d0958da27ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.044Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-dfb0f19c2a634c46e33e64b9: done: 0.057036 sec","job_status":"done","queue_duration_s":0.420549,"scheduling_latency_s":0.420561,"redis_calls":14,"redis_duration_s":0.001885,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000300597,"exclusive_lock_hold_duration_s":0.042544234,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.057036,"completed_at":"2026-10-04T18:59:04.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 171.560987] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"01e0e0130b231c53e8a3adc3","created_at":"2026-10-04T18:59:04.051Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b58abef758903ceeb20ab613736b0e66","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.055Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-01e0e0130b231c53e8a3adc3: start","job_status":"start","queue_duration_s":0.467058,"scheduling_latency_s":0.467073}1450gitlab # [ 171.561806] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"01e0e0130b231c53e8a3adc3","created_at":"2026-10-04T18:59:04.051Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b58abef758903ceeb20ab613736b0e66","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.055Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-01e0e0130b231c53e8a3adc3: done: 0.126744 sec","job_status":"done","queue_duration_s":0.467058,"scheduling_latency_s":0.467073,"redis_calls":12,"redis_duration_s":0.0018579999999999998,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000344177,"exclusive_lock_hold_duration_s":0.111776497,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.126744,"completed_at":"2026-10-04T18:59:04.649Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 171.605138] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"10abaf711a803725deeb41f7","created_at":"2026-10-04T18:59:04.073Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7df737b93cf253d3d6d544cde9a5d3c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T18:59:04.081Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-10abaf711a803725deeb41f7: start","job_status":"start","queue_duration_s":0.569546,"scheduling_latency_s":0.569563}1452gitlab # [ 171.612830] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.659Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1453gitlab # [ 171.653943] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"10abaf711a803725deeb41f7","created_at":"2026-10-04T18:59:04.073Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7df737b93cf253d3d6d544cde9a5d3c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T18:59:04.081Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-10abaf711a803725deeb41f7: done: 0.012773 sec","job_status":"done","queue_duration_s":0.569546,"scheduling_latency_s":0.569563,"redis_calls":10,"redis_duration_s":0.001372,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199467,"duration_s":0.012773,"completed_at":"2026-10-04T18:59:04.663Z","load_balancing_strategy":"primary","db_duration_s":0.000877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 171.654946] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9b095b2775e1008c7042ac15","created_at":"2026-10-04T18:59:04.088Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"094caccb86ad99607a957c81c167dadf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.094Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9b095b2775e1008c7042ac15: start","job_status":"start","queue_duration_s":0.570411,"scheduling_latency_s":0.570425}1455gitlab # [ 171.666065] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"9b095b2775e1008c7042ac15","created_at":"2026-10-04T18:59:04.088Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"094caccb86ad99607a957c81c167dadf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.094Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-9b095b2775e1008c7042ac15: done: 0.010939 sec","job_status":"done","queue_duration_s":0.570411,"scheduling_latency_s":0.570425,"redis_calls":10,"redis_duration_s":0.00118,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196673,"duration_s":0.010939,"completed_at":"2026-10-04T18:59:04.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000235,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 171.706102] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1da78c1821ade908526708fd","created_at":"2026-10-04T18:59:04.105Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acbe4b66f8a43ae6eb092dd43b62a8b7","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.115Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1da78c1821ade908526708fd: start","job_status":"start","queue_duration_s":0.560628,"scheduling_latency_s":0.560641}1457gitlab # [ 171.716694] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1da78c1821ade908526708fd","created_at":"2026-10-04T18:59:04.105Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acbe4b66f8a43ae6eb092dd43b62a8b7","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.115Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1da78c1821ade908526708fd: done: 0.019345 sec","job_status":"done","queue_duration_s":0.560628,"scheduling_latency_s":0.560641,"redis_calls":13,"redis_duration_s":0.00168,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006207772,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019345,"completed_at":"2026-10-04T18:59:04.695Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00185,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1458gitlab # [ 171.757571] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"32e59f242d10649b7a270781","created_at":"2026-10-04T18:59:04.125Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c53355efd6de99b2cad472adb6d68ed9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.132Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-32e59f242d10649b7a270781: start","job_status":"start","queue_duration_s":0.564234,"scheduling_latency_s":0.564248}1459gitlab # [ 171.801866] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.707Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1460gitlab # [ 171.802620] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.713Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"32e59f242d10649b7a270781","created_at":"2026-10-04T18:59:04.125Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c53355efd6de99b2cad472adb6d68ed9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.132Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-32e59f242d10649b7a270781: done: 0.016852 sec","job_status":"done","queue_duration_s":0.564234,"scheduling_latency_s":0.564248,"redis_calls":10,"redis_duration_s":0.001756,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000244724,"duration_s":0.016852,"completed_at":"2026-10-04T18:59:04.713Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 171.811721] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0f434f96035f902c559b2d20","created_at":"2026-10-04T18:59:04.137Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb9c6c6971f4fdadbfc8152253ff4863","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.144Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0f434f96035f902c559b2d20: start","job_status":"start","queue_duration_s":0.569776,"scheduling_latency_s":0.569792}1462gitlab # [ 171.851713] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0f434f96035f902c559b2d20","created_at":"2026-10-04T18:59:04.137Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb9c6c6971f4fdadbfc8152253ff4863","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.144Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0f434f96035f902c559b2d20: done: 0.260041 sec","job_status":"done","queue_duration_s":0.569776,"scheduling_latency_s":0.569792,"redis_calls":565,"redis_duration_s":0.062536,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.061485,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.219001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230477,"duration_s":0.260041,"completed_at":"2026-10-04T18:59:04.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 171.897694] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4ca479e90a5e147bf3bfdd8b","created_at":"2026-10-04T18:59:04.149Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f02370f8c26351f25df2f8786c3d5cb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.154Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4ca479e90a5e147bf3bfdd8b: start","job_status":"start","queue_duration_s":0.820872,"scheduling_latency_s":0.820887}1464gitlab # [ 171.898547] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:04.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4ca479e90a5e147bf3bfdd8b","created_at":"2026-10-04T18:59:04.149Z","trace_propagation_headers":{"sentry-trace":"ba202dbd98b54cab9f9b2b82a3876610-9b7c1daaffbc4f7f","baggage":"sentry-trace_id=ba202dbd98b54cab9f9b2b82a3876610,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f02370f8c26351f25df2f8786c3d5cb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:04.154Z","job_size_bytes":2,"pid":1662,"sidekiq_tid":"maei","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4ca479e90a5e147bf3bfdd8b: done: 0.01838 sec","job_status":"done","queue_duration_s":0.820872,"scheduling_latency_s":0.820887,"redis_calls":12,"redis_duration_s":0.001684,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000340267,"exclusive_lock_hold_duration_s":0.0048556450000000004,"duration_s":0.01838,"completed_at":"2026-10-04T18:59:04.994Z","load_balancing_strategy":"primary","db_duration_s":0.002527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 172.095784] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3657375452 wd_nsec: 36573752721466gitlab # [ 189.009943] postgres[2474]: [2474] ERROR: relation "table_without_models" does not exist1467gitlab # [ 189.011634] postgres[2474]: [2474] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1468gitlab # [ 191.873627] runuser[2275]: pam_unix(runuser:session): session closed for user gitlab1469gitlab # [ 191.875199] setup-gitlab-start[2257]: + systemctl restart gitlab-sidekiq.service1470gitlab # [ 191.901947] systemd[1]: Stopping gitlab-sidekiq.service...1471gitlab # [ 191.904694] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:26.912Z","message":"Shutting down"}1472gitlab # [ 191.907396] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:26.914Z","message":"Scheduler exiting..."}1473gitlab # [ 191.910182] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:26.916Z","message":"Terminating quiet threads for default capsule"}1474gitlab # [ 191.913394] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:26.917Z","message":"Scheduler exiting..."}1475gitlab # [ 192.412122] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:27.419Z","message":"Pausing to allow jobs to finish..."}1476gitlab # [ 195.417481] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:30.424Z","message":"Bye!"}1477gitlab # [ 195.441758] sidekiq-cluster[1662]: {"severity":"INFO","time":"2026-10-04T18:59:30.449Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1662,"worker_id":"sidekiq_0","memwd_rss_bytes":934031360,"retry":0}1478gitlab # [ 196.412314] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1479gitlab # [ 196.415007] systemd[1]: Stopped gitlab-sidekiq.service.1480gitlab # [ 196.416834] systemd[1]: gitlab-sidekiq.service: Consumed 59.532s CPU time over 1min 58.315s wall clock time, 965.6M memory peak, 7M read from disk.1481gitlab # [ 196.425529] systemd[1]: Started gitlab-sidekiq.service.1482gitlab # [ 196.429649] setup-gitlab-start[2257]: + mkdir -p /var/lib/secrets1483gitlab # [ 196.434972] setup-gitlab-start[2257]: + echo nixbot-test-token-of-20-chars1484gitlab # [ 196.435853] setup-gitlab-start[2257]: + chmod 644 /var/lib/secrets/gitlab-token1485gitlab # [ 196.442930] setup-gitlab-start[2257]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1486gitlab # [ 196.687976] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:31 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 2041487gitlab # [ 197.204395] gitaly[915]: time="2026-10-04T18:59:32.212Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.211" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1488gitlab # [ 197.204661] gitaly[915]: time="2026-10-04T18:59:32.212Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.211" grpc.time_ms=0.496 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 197.340608] gitaly[915]: time="2026-10-04T18:59:32.348Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.137" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.348" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 197.364066] gitaly[915]: time="2026-10-04T18:59:32.348Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.137" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.348" grpc.time_ms=0.519 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1491gitlab # [ 197.441763] gitaly[915]: time="2026-10-04T18:59:32.449Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS diskcache=7fb5fbaa-eb47-4d98-bd6c-b7e823e32d8c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:59:42.187" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.398" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 197.442226] gitaly[915]: time="2026-10-04T18:59:32.450Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1573 command.oublock=32 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:59:42.187" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.398" grpc.time_ms=51.552 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1493gitlab # [ 197.492224] gitaly[915]: time="2026-10-04T18:59:32.500Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.288" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.499" pid=915 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1494gitlab # [ 197.493088] gitaly[915]: time="2026-10-04T18:59:32.500Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.288" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.499" grpc.time_ms=1.164 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1495gitlab # [ 197.527793] gitaly[915]: time="2026-10-04T18:59:32.535Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:59:42.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.525" grpc.time_ms=10.38 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1496gitlab # [ 197.614171] gitaly[915]: time="2026-10-04T18:59:32.622Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.100" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.621" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1497gitlab # [ 197.636949] gitaly[915]: time="2026-10-04T18:59:32.644Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.100" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.621" grpc.time_ms=0.536 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1498gitlab # [ 197.709575] gitaly[915]: time="2026-10-04T18:59:32.717Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS diskcache=fab6c1ca-8a46-4062-9074-e4bfed501c7a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:59:42.148" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.669" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1499gitlab # [ 197.710229] gitaly[915]: time="2026-10-04T18:59:32.717Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1580 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:59:42.148" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.669" grpc.time_ms=48.198 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1500gitlab # [ 197.748212] gitaly[915]: time="2026-10-04T18:59:32.756Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.234" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.755" pid=915 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1501gitlab # [ 197.749090] gitaly[915]: time="2026-10-04T18:59:32.756Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.234" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.755" grpc.time_ms=0.861 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1502gitlab # [ 197.784569] gitaly[915]: time="2026-10-04T18:59:32.792Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:59:42.260" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.781" grpc.time_ms=11.173 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1503gitlab # [ 197.836103] gitaly[915]: time="2026-10-04T18:59:32.843Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:59:42.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.843" grpc.time_ms=0.557 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1504gitlab # [ 197.863938] gitaly[915]: time="2026-10-04T18:59:32.871Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T18:59:42.031" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:32.871" grpc.time_ms=0.304 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1505gitlab # [ 198.377868] gitaly[915]: time="2026-10-04T18:59:33.384Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444J785CHJ8AHV41S70JAMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:59:43.162" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:33.373" grpc.time_ms=11.069 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1506gitlab # [ 198.614244] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:33 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 19031507gitlab # [ 198.618866] setup-gitlab-start[2529]: {"id":1,"message":null,"error":null}1508gitlab # [ 199.583677] sidekiq-cluster[2513]: {"severity":"INFO","time":"2026-10-04T18:59:34.574Z","message":"Starting cluster with 1 processes"}1509gitlab # [ 203.761450] gitaly[915]: time="2026-10-04T18:59:38.769Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444JE0T6VJQ9ME0T0DBN8VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:59:48.236" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:38.757" grpc.time_ms=11.258 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1510gitlab # [ 203.877546] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:38 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 2341511gitlab # [ 203.885284] setup-gitlab-start[2257]: + mkdir -p /tmp/test-flake1512gitlab # [ 203.889873] setup-gitlab-start[2257]: + cd /tmp/test-flake1513gitlab # [ 203.890564] setup-gitlab-start[2257]: + git init -b master1514gitlab # [ 203.922673] setup-gitlab-start[2602]: Initialized empty Git repository in /tmp/test-flake/.git/1515gitlab # [ 203.925413] setup-gitlab-start[2257]: + git config user.name test1516gitlab # [ 203.931853] setup-gitlab-start[2257]: + git config user.email test@example.com1517gitlab # [ 203.939283] setup-gitlab-start[2257]: + cat1518gitlab # [ 203.944283] setup-gitlab-start[2257]: + git add flake.nix1519gitlab # [ 203.951682] setup-gitlab-start[2257]: + git commit -m 'initial commit'1520gitlab # [ 203.979594] setup-gitlab-start[2607]: [master (root-commit) 66b3c13] initial commit1521gitlab # [ 203.980327] setup-gitlab-start[2607]: 1 file changed, 10 insertions(+)1522gitlab # [ 203.981416] setup-gitlab-start[2607]: create mode 100644 flake.nix1523gitlab # [ 203.986482] setup-gitlab-start[2257]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1524gitlab # [ 203.998112] setup-gitlab-start[2257]: + git push -u origin master1525gitlab # [ 204.088578] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 531526gitlab # [ 204.188974] gitaly[915]: time="2026-10-04T18:59:39.196Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEEYB6RAF37BV52H4CA4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:39.185" grpc.time_ms=11.279 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1527gitlab # [ 204.217643] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1221528gitlab # [ 204.931664] gitaly[915]: time="2026-10-04T18:59:39.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:59:49.086" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:39.927" grpc.time_ms=10.901 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1529gitlab # [ 204.995094] gitaly[915]: time="2026-10-04T18:59:40.002Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T19:00:09.089" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T18:59:39.990" grpc.time_ms=12.693 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1530gitlab # [ 205.030466] gitaly[915]: time="2026-10-04T18:59:40.038Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M444JEK6V14KGP481M807Z43 duration_ms=163ns method=POST pid=915 status=200 url="http://unix/api/v4/internal/allowed"1531gitlab # [ 205.054298] gitaly[915]: time="2026-10-04T18:59:40.062Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M444JEK6V14KGP481M807Z43 duration_ms=22ns method=POST pid=915 status=200 url="http://unix/api/v4/internal/pre_receive"1532gitlab # [ 205.061317] gitaly[915]: time="2026-10-04T18:59:40.069Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:39.874" grpc.time_ms=194.924 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1533gitlab # [ 205.125249] gitaly[915]: time="2026-10-04T18:59:40.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:40.130" grpc.time_ms=2.172 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1534gitlab # [ 205.172517] gitaly[915]: time="2026-10-04T18:59:40.180Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:40.179" grpc.time_ms=0.614 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1535gitlab # [ 205.219114] gitaly[915]: time="2026-10-04T18:59:40.227Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:40.226" grpc.time_ms=0.623 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1536gitlab # [ 205.303817] gitaly[915]: time="2026-10-04T18:59:40.311Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:59:50.095" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:40.305" grpc.time_ms=5.752 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 205.338846] gitaly[915]: time="2026-10-04T18:59:40.346Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T18:59:50.123" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:40.333" grpc.time_ms=12.632 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1538gitlab # [ 205.362340] gitaly[915]: time="2026-10-04T18:59:40.370Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M444JEK6V14KGP481M807Z43 duration_ms=112ns method=POST pid=915 status=200 url="http://unix/api/v4/internal/post_receive"1539gitlab # [ 205.365618] gitaly[915]: time="2026-10-04T18:59:40.373Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:40.257" grpc.time_ms=116.105 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1540gitlab # [ 205.386922] gitaly[915]: time="2026-10-04T18:59:40.394Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 diskcache=0351b557-18a4-46f3-8915-06229ca06ace grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:39.795" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1541gitlab # [ 205.396962] gitlab-workhorse[826]: time="2026-10-04T18:59:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521542gitlab # [ 205.397760] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11651543gitlab # [ 205.398584] gitaly[915]: time="2026-10-04T18:59:40.395Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=227 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13975 command.oublock=32 command.real_time_ms=589 command.spawn_token_fork_ms=0 command.system_time_ms=141 command.user_time_ms=86 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:39.795" grpc.time_ms=599.508 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1544gitlab # [ 205.414693] setup-gitlab-start[2611]: To http://gitlab/root/test-flake.git1545gitlab # [ 205.415374] setup-gitlab-start[2611]: * [new branch] master -> master1546gitlab # [ 205.420863] setup-gitlab-start[2611]: branch 'master' set up to track 'origin/master'.1547gitlab # [ 205.424132] systemd[1]: Finished setup-gitlab.service.1548gitlab # [ 205.429715] systemd[1]: Starting nixbot CI service...1549gitlab: (finished: waiting for unit setup-gitlab.service, in 206.62 seconds)1550gitlab: waiting for unit nixbot.service1551gitlab # [ 206.796547] nixbot[2691]: {"ts": "2026-10-04T18:59:41.803Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1552gitlab # [ 206.834512] nixbot[2691]: {"ts": "2026-10-04T18:59:41.842Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1553gitlab # [ 206.849749] nixbot[2691]: {"ts": "2026-10-04T18:59:41.857Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1554gitlab # [ 206.855392] nixbot[2691]: {"ts": "2026-10-04T18:59:41.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1555gitlab # [ 206.860738] nixbot[2691]: {"ts": "2026-10-04T18:59:41.868Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1556gitlab # [ 206.867262] nixbot[2691]: {"ts": "2026-10-04T18:59:41.875Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1557gitlab # [ 206.873961] nixbot[2691]: {"ts": "2026-10-04T18:59:41.881Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1558gitlab # [ 206.879458] nixbot[2691]: {"ts": "2026-10-04T18:59:41.887Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1559gitlab # [ 206.886533] nixbot[2691]: {"ts": "2026-10-04T18:59:41.894Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1560gitlab # [ 206.890694] nixbot[2691]: {"ts": "2026-10-04T18:59:41.898Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1561gitlab # [ 206.896852] nixbot[2691]: {"ts": "2026-10-04T18:59:41.904Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1562gitlab # [ 206.906449] nixbot[2691]: {"ts": "2026-10-04T18:59:41.914Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1563gitlab # [ 206.911443] nixbot[2691]: {"ts": "2026-10-04T18:59:41.919Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1564gitlab # [ 206.915581] nixbot[2691]: {"ts": "2026-10-04T18:59:41.923Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1565gitlab # [ 206.920844] nixbot[2691]: {"ts": "2026-10-04T18:59:41.928Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1566gitlab # [ 206.925126] nixbot[2691]: {"ts": "2026-10-04T18:59:41.933Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1567gitlab # [ 206.929235] nixbot[2691]: {"ts": "2026-10-04T18:59:41.937Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1568gitlab # [ 206.933339] nixbot[2691]: {"ts": "2026-10-04T18:59:41.941Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1569gitlab # [ 206.941322] nixbot[2691]: {"ts": "2026-10-04T18:59:41.949Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1570gitlab # [ 206.945814] nixbot[2691]: {"ts": "2026-10-04T18:59:41.953Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1571gitlab # [ 206.950169] nixbot[2691]: {"ts": "2026-10-04T18:59:41.958Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1572gitlab # [ 206.958865] nixbot[2691]: {"ts": "2026-10-04T18:59:41.966Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1573gitlab # [ 206.963156] nixbot[2691]: {"ts": "2026-10-04T18:59:41.971Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1574gitlab # [ 206.968125] nixbot[2691]: {"ts": "2026-10-04T18:59:41.976Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1575gitlab # [ 206.972775] nixbot[2691]: {"ts": "2026-10-04T18:59:41.980Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1576gitlab # [ 206.977502] nixbot[2691]: {"ts": "2026-10-04T18:59:41.985Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1577gitlab # [ 206.981898] nixbot[2691]: {"ts": "2026-10-04T18:59:41.989Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1578gitlab # [ 206.986770] nixbot[2691]: {"ts": "2026-10-04T18:59:41.994Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1579gitlab # [ 206.991095] nixbot[2691]: {"ts": "2026-10-04T18:59:41.998Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1580gitlab # [ 207.007130] nixbot[2691]: {"ts": "2026-10-04T18:59:42.014Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1581gitlab # [ 207.016169] nixbot[2691]: {"ts": "2026-10-04T18:59:42.023Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1582gitlab # [ 207.018934] nixbot[2691]: {"ts": "2026-10-04T18:59:42.026Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1583gitlab # [ 207.022572] nixbot[2691]: {"ts": "2026-10-04T18:59:42.030Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1584gitlab # [ 207.031566] nixbot[2691]: {"ts": "2026-10-04T18:59:42.038Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1585gitlab # [ 207.034627] nixbot[2691]: {"ts": "2026-10-04T18:59:42.042Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1586gitlab # [ 207.041881] nixbot[2691]: {"ts": "2026-10-04T18:59:42.049Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1587gitlab # [ 207.047775] nixbot[2691]: {"ts": "2026-10-04T18:59:42.055Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1588gitlab # [ 207.468900] nixbot[2691]: INFO: Started server process [2691]1589gitlab # [ 207.471443] nixbot[2691]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1590gitlab # [ 207.509729] nixbot[2691]: INFO: ::ffff:127.0.0.1:45686 - "GET /health HTTP/1.1" 200 OK1591gitlab # [ 207.515792] systemd[1]: Started nixbot CI service.1592gitlab # [ 207.517883] systemd[1]: Reached target Multi-User System.1593gitlab # [ 207.519119] systemd[1]: Startup finished in 1.284s (kernel) + 3.625s (initrd) + 3min 22.609s (userspace) = 3min 27.518s.1594gitlab # [ 207.725666] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:42 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2141595gitlab # [ 207.731380] nixbot[2691]: {"ts": "2026-10-04T18:59:42.739Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1596gitlab # [ 207.764589] nixbot[2691]: {"ts": "2026-10-04T18:59:42.772Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1597gitlab # [ 207.886324] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1071598gitlab # [ 207.888490] nixbot[2691]: {"ts": "2026-10-04T18:59:42.896Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1599gitlab # [ 207.890528] nixbot[2691]: {"ts": "2026-10-04T18:59:42.898Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1600gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1601gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1602gitlab # [ 208.097202] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1941603gitlab # [ 208.103152] nixbot[2691]: {"ts": "2026-10-04T18:59:43.110Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1604gitlab # [ 208.112896] nixbot[2691]: INFO: ::ffff:127.0.0.1:45694 - "GET /health HTTP/1.1" 200 OK1605gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds)1606subtest: project discovered and webhook registered1607gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1608gitlab # [ 208.314595] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 1651609gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.20 seconds)1610(finished: subtest: project discovered and webhook registered, in 0.20 seconds)1611subtest: push delivers the webhook1612gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1613gitlab # [ 208.757889] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 6351614gitlab # [ 208.759079] nixbot[2691]: {"ts": "2026-10-04T18:59:43.766Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1615gitlab # [ 208.836129] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 631616gitlab # [ 208.837427] nixbot[2691]: {"ts": "2026-10-04T18:59:43.845Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1617gitlab # [ 208.925564] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:43 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 821618gitlab # [ 208.931273] nixbot[2691]: {"ts": "2026-10-04T18:59:43.934Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1619gitlab # [ 209.013301] gitaly[915]: time="2026-10-04T18:59:44.020Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JK65RWQKNG4ZH3HCEYPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-04T19:00:14.007" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:44.008" grpc.time_ms=12.018 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1620gitlab # [ 209.046436] gitaly[915]: time="2026-10-04T18:59:44.032Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JK65RWQKNG4ZH3HCEYPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:14.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:59:44.031" grpc.time_ms=0.666 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1621gitlab # [ 209.070257] gitaly[915]: time="2026-10-04T18:59:44.040Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JK65RWQKNG4ZH3HCEYPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-04T18:59:54.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:59:44.034" grpc.time_ms=6.101 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1622gitlab # [ 209.082588] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1211623gitlab # [ 209.085134] nixbot[2691]: {"ts": "2026-10-04T18:59:44.078Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1624gitlab # [ 209.196367] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1021625gitlab # [ 209.202166] nixbot[2691]: {"ts": "2026-10-04T18:59:44.202Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1626gitlab # [ 209.212408] nixbot[2691]: {"ts": "2026-10-04T18:59:44.220Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1627gitlab # [ 209.392300] gitaly[915]: time="2026-10-04T18:59:44.400Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444JKHYXTV76ZZ8NWV3CNRV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.388" grpc.time_ms=11.582 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1628gitlab # [ 209.417177] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 821629gitlab # [ 209.478180] gitaly[915]: time="2026-10-04T18:59:44.486Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444JKMJ5DQ5J7CYE0M55M1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.473" pid=915 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1630gitlab # [ 209.499795] gitaly[915]: time="2026-10-04T18:59:44.486Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444JKMJ5DQ5J7CYE0M55M1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.473" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1631gitlab # [ 209.520314] gitaly[915]: time="2026-10-04T18:59:44.528Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444JKMJ5DQ5J7CYE0M55M1Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.473" grpc.time_ms=54.233 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1632gitlab # [ 209.521261] gitlab-workhorse[826]: time="2026-10-04T18:59:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131633gitlab # [ 209.521926] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1261634gitlab # [ 209.597675] gitaly[915]: time="2026-10-04T18:59:44.605Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444JKRPHNZSK6TXNRQJA48Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.586" pid=915 remote_ip="2001:db8:1::1" request_sha=dc33b6aed6ab98067ebe906172db756e3dc34150 response_bytes=223 span.kind=server system=grpc1635gitlab # [ 209.615415] gitaly[915]: time="2026-10-04T18:59:44.605Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444JKRPHNZSK6TXNRQJA48Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.586" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1636gitlab # [ 209.628858] gitaly[915]: time="2026-10-04T18:59:44.636Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444JKRPHNZSK6TXNRQJA48Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.586" grpc.time_ms=50.204 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1637gitlab # [ 209.631252] gitlab-workhorse[826]: time="2026-10-04T18:59:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231638gitlab # [ 209.631970] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1031639gitlab # [ 209.726336] gitaly[915]: time="2026-10-04T18:59:44.733Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444JKXDV8WXYM7E8WHNTMFA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.722" grpc.time_ms=10.669 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1640gitlab # [ 209.726819] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 481641gitlab # [ 209.873336] gitaly[915]: time="2026-10-04T18:59:44.881Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444JKZV09YKZ7YRE9TBWK3G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.867" pid=915 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1642gitlab # [ 209.873609] gitaly[915]: time="2026-10-04T18:59:44.881Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JKZV09YKZ7YRE9TBWK3G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.867" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1643gitlab # [ 209.920910] gitaly[915]: time="2026-10-04T18:59:44.928Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JKZV09YKZ7YRE9TBWK3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:44.867" grpc.time_ms=61.348 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1644gitlab # [ 209.937002] gitlab-workhorse[826]: time="2026-10-04T18:59:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671645gitlab # [ 209.937665] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1801646gitlab # [ 210.055098] gitaly[915]: time="2026-10-04T18:59:45.061Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444JM6YV5ST48HMVW5136YW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:45.049" grpc.time_ms=11.855 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1647gitlab # [ 210.056090] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 711648gitlab # [ 210.182107] gitaly[915]: time="2026-10-04T18:59:45.190Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444JMA9XH1917DAEMJAMT99 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:45.166" pid=915 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1649gitlab # [ 210.182389] gitaly[915]: time="2026-10-04T18:59:45.190Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMA9XH1917DAEMJAMT99 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:45.166" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1650gitlab # [ 210.183580] gitaly[915]: time="2026-10-04T18:59:45.190Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMA9XH1917DAEMJAMT99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:59:45.166" grpc.time_ms=23.914 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1651gitlab # [ 210.184769] gitlab-workhorse[826]: time="2026-10-04T18:59:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091652gitlab # [ 210.185868] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 931653gitlab # [ 210.352158] gitaly[915]: time="2026-10-04T18:59:45.359Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:15.357" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:59:45.358" grpc.time_ms=0.941 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654gitlab # [ 210.409872] gitaly[915]: time="2026-10-04T18:59:45.417Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T19:00:15.410" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:45.410" grpc.time_ms=6.703 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1655gitlab # [ 210.431838] gitaly[915]: time="2026-10-04T18:59:45.439Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:59:55.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:45.429" grpc.time_ms=10.503 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1656gitlab # [ 210.448879] gitaly[915]: time="2026-10-04T18:59:45.456Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:59:55.025" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:45.445" grpc.time_ms=10.677 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1657gitlab # [ 210.530405] gitaly[915]: time="2026-10-04T18:59:45.538Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T18:59:55.116" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:45.537" grpc.time_ms=0.527 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1658gitlab # [ 210.565915] gitaly[915]: time="2026-10-04T18:59:45.573Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T18:59:55.142" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:45.562" grpc.time_ms=10.998 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1659gitlab # [ 210.599275] gitaly[915]: time="2026-10-04T18:59:45.607Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:45.606" grpc.time_ms=0.531 pid=915 span.kind=server system=grpc1660gitlab # [ 210.636218] gitaly[915]: time="2026-10-04T18:59:45.644Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:59:45.643" grpc.time_ms=0.377 pid=915 span.kind=server system=grpc1661gitlab # [ 210.658722] gitaly[915]: time="2026-10-04T18:59:45.666Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 diskcache=b4579f0f-d658-4483-a8c2-8d7bbac13c39 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:59:55.155" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:45.576" pid=915 ref=refs/keep-around/66b3c13412ea8aafe07a6f1c588cbcbdda058456 remote_ip="2001:db8:1::1" revision=66b3c13412ea8aafe07a6f1c588cbcbdda058456 span.kind=server system=grpc username=root1662gitlab # [ 210.669437] gitaly[915]: time="2026-10-04T18:59:45.667Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=55 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5307 command.oublock=8 command.real_time_ms=80 command.spawn_token_fork_ms=1 command.system_time_ms=36 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:59:55.155" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:59:45.576" grpc.time_ms=90.683 pid=915 ref=refs/keep-around/66b3c13412ea8aafe07a6f1c588cbcbdda058456 remote_ip="2001:db8:1::1" revision=66b3c13412ea8aafe07a6f1c588cbcbdda058456 span.kind=server system=grpc username=root1663gitlab # [ 210.670193] gitaly[915]: time="2026-10-04T18:59:45.672Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:15.671" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:59:45.671" grpc.time_ms=0.36 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 210.794423] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5001665gitlab # [ 210.796817] nixbot[2691]: {"ts": "2026-10-04T18:59:45.804Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1666gitlab # [ 210.808169] nixbot[2691]: {"ts": "2026-10-04T18:59:45.816Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-66b3c13412ea-0"}1667gitlab # [ 211.041264] nixbot[2691]: {"ts": "2026-10-04T18:59:46.049Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-66b3c13412ea-0", "argv0": "bwrap"}1668gitlab # [ 211.145800] systemd[1]: Started Nix Daemon.1669gitlab # [ 211.208733] nix-daemon[2863]: accepted connection from pid 2858, user nixbot1670gitlab # [ 215.841978] nix-daemon[2863]: accepted connection from pid 2912, user nixbot1671gitlab # [ 215.860612] nix-daemon[2863]: accepted connection from pid 2858, user nixbot1672gitlab # [ 220.539097] nix-daemon[2863]: accepted connection from pid 2858, user nixbot1673gitlab # [ 220.638478] nix-daemon[2863]: accepted connection from pid 2954, user nixbot1674gitlab # [ 220.685579] gitaly[915]: time="2026-10-04T18:59:55.692Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JYKJWW5J1SXT1A4WM1AH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:25.690" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:59:55.691" grpc.time_ms=0.922 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675gitlab # [ 220.714378] gitaly[915]: time="2026-10-04T18:59:55.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JYKJWW5J1SXT1A4WM1AH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:05.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:55.712" grpc.time_ms=7.146 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1676gitlab # [ 220.748548] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1201677gitlab # [ 220.750467] nixbot[2691]: {"ts": "2026-10-04T18:59:55.757Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1678gitlab # [ 220.803817] gitaly[915]: time="2026-10-04T18:59:55.811Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JYQPAWK2V8GNAM60XR94 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:25.807" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:59:55.808" grpc.time_ms=2.09 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679gitlab # [ 220.845211] gitaly[915]: time="2026-10-04T18:59:55.852Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444JYQPAWK2V8GNAM60XR94 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:05.135" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:59:55.845" grpc.time_ms=7.436 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680gitlab # [ 220.879215] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:59:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1201681gitlab # [ 220.881992] nixbot[2691]: {"ts": "2026-10-04T18:59:55.888Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1682gitlab # [ 225.231073] sidekiq-cluster[2570]: 2026-10-04T19:00:00.238Z pid=2570 tid=xb6 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2570, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>449515520}1683gitlab # [ 225.401651] sidekiq-cluster[2570]: W, [2026-10-04T19:00:00.409409 #2570] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1684gitlab # [ 225.440142] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:00.447Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1685gitlab # [ 225.478854] systemd[1]: Starting Logrotate Service...1686gitlab # [ 225.555437] systemd[1]: logrotate.service: Deactivated successfully.1687gitlab # [ 225.557816] systemd[1]: Finished Logrotate Service.1688gitlab # [ 225.580182] nix-daemon[2863]: accepted connection from pid 3032, user nixbot1689gitlab # [ 225.661546] gitaly[915]: time="2026-10-04T19:00:00.668Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444K3F81HZ5DDWRKQ42620P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:30.663" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:00.664" grpc.time_ms=3.934 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690gitlab # [ 225.690338] gitaly[915]: time="2026-10-04T19:00:00.694Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444K3F81HZ5DDWRKQ42620P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:10.269" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:00.689" grpc.time_ms=5.271 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1691gitlab # [ 225.719802] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1101692gitlab # [ 225.726927] nixbot[2691]: {"ts": "2026-10-04T19:00:00.728Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1693gitlab # [ 226.013731] gitaly[915]: time="2026-10-04T19:00:01.021Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444K3SS31QBH2CSAYDDJ1M4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.011" grpc.time_ms=9.379 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1694gitlab # [ 226.028921] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 681695gitlab # [ 226.082427] gitaly[915]: time="2026-10-04T19:00:01.090Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444K3WC15SB5DADJ2ZZCWWP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.080" pid=915 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1696gitlab # [ 226.100368] gitaly[915]: time="2026-10-04T19:00:01.090Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444K3WC15SB5DADJ2ZZCWWP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.080" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1697gitlab # [ 226.113338] gitlab-workhorse[826]: time="2026-10-04T19:00:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671698gitlab # [ 226.115167] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 551699gitlab # [ 226.117121] gitaly[915]: time="2026-10-04T19:00:01.090Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444K3WC15SB5DADJ2ZZCWWP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.080" grpc.time_ms=9.767 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1700gitlab # [ 226.195861] gitaly[915]: time="2026-10-04T19:00:01.203Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444K4075V3J5T1FM89M3T4T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.197" grpc.time_ms=6.412 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1701gitlab # [ 226.199647] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 471702gitlab # [ 226.251836] gitaly[915]: time="2026-10-04T19:00:01.259Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444K421XAMSA9JQXXGBVTYF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.252" pid=915 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1703gitlab # [ 226.252273] gitaly[915]: time="2026-10-04T19:00:01.260Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444K421XAMSA9JQXXGBVTYF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.252" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1704gitlab # [ 226.253277] gitaly[915]: time="2026-10-04T19:00:01.261Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444K421XAMSA9JQXXGBVTYF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:01.252" grpc.time_ms=9.026 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1705gitlab # [ 226.254272] gitlab-workhorse[826]: time="2026-10-04T19:00:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671706gitlab # [ 226.254951] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 451707gitlab # [ 227.585302] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:02.592Z","message":"GitLab reliable fetch activated!"}1708gitlab # [ 227.585656] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:02.592Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1709gitlab # [ 248.261762] postgres[3262]: [3262] ERROR: relation "table_without_models" does not exist1710gitlab # [ 248.262293] postgres[3262]: [3262] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1711gitlab # [ 250.734852] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:25.738Z","message":"Booted Rails 7.2.3.1 application in production environment"}1712gitlab # [ 250.735560] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:25.739Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1713gitlab # [ 250.736240] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:25.739Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1714gitlab # [ 250.736862] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:25.739Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1715gitlab # [ 250.740192] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:25.740Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1716gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 42.62 seconds)1717gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1718gitlab # [ 251.043602] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 471719gitlab # [ 251.141661] gitaly[915]: time="2026-10-04T19:00:26.148Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444KWA49H9QYEJ5T3G71K4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:26.135" grpc.time_ms=12.82 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1720gitlab # [ 251.174099] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1291721gitlab # Enumerating objects: 5, done.1722gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1723gitlab # Delta compression using up to 4 threads1724gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1725gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1726gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1727gitlab # [ 251.427254] gitaly[915]: time="2026-10-04T19:00:26.435Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T19:00:56.022" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T19:00:26.423" grpc.time_ms=11.163 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728gitlab # [ 251.468210] gitaly[915]: time="2026-10-04T19:00:26.476Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M444KWF7WKSJSNAN1WEK0STC duration_ms=114ns method=POST pid=915 status=200 url="http://unix/api/v4/internal/allowed"1729gitlab # [ 251.496666] gitaly[915]: time="2026-10-04T19:00:26.504Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M444KWF7WKSJSNAN1WEK0STC duration_ms=27ns method=POST pid=915 status=200 url="http://unix/api/v4/internal/pre_receive"1730gitlab # [ 251.497172] gitaly[915]: time="2026-10-04T19:00:26.505Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:26.360" grpc.time_ms=144.567 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1731gitlab # [ 251.567698] gitaly[915]: time="2026-10-04T19:00:26.575Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:26.574" grpc.time_ms=0.79 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1732gitlab # [ 251.633568] gitaly[915]: time="2026-10-04T19:00:26.641Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:26.640" grpc.time_ms=0.667 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1733gitlab # [ 251.680607] gitaly[915]: time="2026-10-04T19:00:26.688Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:26.687" grpc.time_ms=0.583 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1734gitlab # [ 251.762688] gitaly[915]: time="2026-10-04T19:00:26.770Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M444KWF7WKSJSNAN1WEK0STC duration_ms=41ns method=POST pid=915 status=200 url="http://unix/api/v4/internal/post_receive"1735gitlab # [ 251.766952] gitaly[915]: time="2026-10-04T19:00:26.773Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:26.728" grpc.time_ms=45.745 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1736gitlab # [ 251.791833] gitaly[915]: time="2026-10-04T19:00:26.799Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC diskcache=9e020e3d-ef88-4913-99ed-0daad13904ff grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:26.290" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1737gitlab # [ 251.792828] gitlab-workhorse[826]: time="2026-10-04T19:00:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521738gitlab # [ 251.793614] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:26 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 5851739gitlab # [ 251.794114] gitaly[915]: time="2026-10-04T19:00:26.800Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=228 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13583 command.oublock=32 command.real_time_ms=500 command.spawn_token_fork_ms=0 command.system_time_ms=142 command.user_time_ms=86 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:26.290" grpc.time_ms=509.191 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1740gitlab # To http://gitlab/root/test-flake.git1741gitlab # 66b3c13..7edc24b master -> master1742gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.89 seconds)1743gitlab: must succeed: git -C /tmp/test-flake rev-parse master1744gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1745gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1746gitlab # [ 251.904885] nixbot[2691]: INFO: ::ffff:127.0.0.1:58860 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1747gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.08 seconds)1748gitlab # [ 252.217949] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.225Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"82014a46db4423dd6bc3f298","created_at":"2026-10-04T18:59:32.561Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:32.595Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-82014a46db4423dd6bc3f298: start","job_status":"start","queue_duration_s":54.628976,"scheduling_latency_s":54.629022}1749gitlab # [ 252.382833] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.390Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"82014a46db4423dd6bc3f298","created_at":"2026-10-04T18:59:32.561Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:32.595Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-82014a46db4423dd6bc3f298: done: 0.165499 sec","job_status":"done","queue_duration_s":54.628976,"scheduling_latency_s":54.629022,"redis_calls":25,"redis_duration_s":0.006239,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003517,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001509,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001213,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.086225205,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.165499,"completed_at":"2026-10-04T19:00:27.390Z","load_balancing_strategy":"primary","db_duration_s":0.06148,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1750gitlab # [ 252.384646] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.392Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7f7262a2f6e2d1bf8ca6a837","created_at":"2026-10-04T18:59:32.604Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:32.614Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7f7262a2f6e2d1bf8ca6a837: start","job_status":"start","queue_duration_s":54.778054,"scheduling_latency_s":54.77807}1751gitlab # [ 252.469338] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.476Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7f7262a2f6e2d1bf8ca6a837","created_at":"2026-10-04T18:59:32.604Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:32.614Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7f7262a2f6e2d1bf8ca6a837: done: 0.084892 sec","job_status":"done","queue_duration_s":54.778054,"scheduling_latency_s":54.77807,"redis_calls":14,"redis_duration_s":0.003516,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001548,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.066978624,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.084892,"completed_at":"2026-10-04T19:00:27.476Z","load_balancing_strategy":"primary","db_duration_s":0.026964,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1752gitlab # [ 252.472981] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.478Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"6a2690ee26e9f5f1d3ee08ad","created_at":"2026-10-04T18:59:33.270Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:33.275Z","job_size_bytes":5,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-6a2690ee26e9f5f1d3ee08ad: start","job_status":"start","queue_duration_s":54.203153,"scheduling_latency_s":54.203169}1753gitlab # [ 252.630107] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.637Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"6a2690ee26e9f5f1d3ee08ad","created_at":"2026-10-04T18:59:33.270Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:33.275Z","job_size_bytes":5,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-6a2690ee26e9f5f1d3ee08ad: done: 0.159254 sec","job_status":"done","queue_duration_s":54.203153,"scheduling_latency_s":54.203169,"redis_calls":12,"redis_duration_s":0.002678,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001675,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000293892,"duration_s":0.159254,"completed_at":"2026-10-04T19:00:27.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027662,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1754gitlab # [ 252.631127] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.639Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f630c1c5b54ef579dc8634ad","created_at":"2026-10-04T18:59:33.296Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:33.300Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f630c1c5b54ef579dc8634ad: start","job_status":"start","queue_duration_s":54.338891,"scheduling_latency_s":54.338906}1755gitlab # [ 252.788936] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.796Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f630c1c5b54ef579dc8634ad","created_at":"2026-10-04T18:59:33.296Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:33.300Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f630c1c5b54ef579dc8634ad: done: 0.157523 sec","job_status":"done","queue_duration_s":54.338891,"scheduling_latency_s":54.338906,"redis_calls":11,"redis_duration_s":0.002697,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000732,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.131,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.131,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340546,"duration_s":0.157523,"completed_at":"2026-10-04T19:00:27.796Z","load_balancing_strategy":"primary","db_duration_s":0.072188,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1756gitlab # [ 252.790412] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.798Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c79f96cee2b8d4724adde112","created_at":"2026-10-04T18:59:33.302Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:33.305Z","job_size_bytes":89,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c79f96cee2b8d4724adde112: start","job_status":"start","queue_duration_s":54.492672,"scheduling_latency_s":54.492687}1757gitlab # [ 252.839758] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.826Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c79f96cee2b8d4724adde112","created_at":"2026-10-04T18:59:33.302Z","trace_propagation_headers":{"sentry-trace":"13c5804d843d4029a391d68f9e942103-795e5f633e8140b8","baggage":"sentry-trace_id=13c5804d843d4029a391d68f9e942103,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444J785CHJ8AHV41S70JAMS","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:33.305Z","job_size_bytes":89,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c79f96cee2b8d4724adde112: done: 0.027946 sec","job_status":"done","queue_duration_s":54.492672,"scheduling_latency_s":54.492687,"redis_calls":11,"redis_duration_s":0.002515,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00074,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000314286,"duration_s":0.027946,"completed_at":"2026-10-04T19:00:27.826Z","load_balancing_strategy":"primary","db_duration_s":0.006044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1758gitlab # [ 252.853969] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.827Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 66b3c13412ea8aafe07a6f1c588cbcbdda058456 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"795db3b9998a0cf97beae68e","created_at":"2026-10-04T18:59:40.284Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-b8058c408d504c04","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:40.292Z","job_size_bytes":185,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-795db3b9998a0cf97beae68e: start","job_status":"start","queue_duration_s":47.535213,"scheduling_latency_s":47.535227}1759gitlab # [ 252.897600] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:27.838Z","message":"changes: \"0000000000000000000000000000000000000000 66b3c13412ea8aafe07a6f1c588cbcbdda058456 refs/heads/master\\n\""}1760gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1761gitlab # [ 252.898680] gitaly[915]: time="2026-10-04T19:00:27.881Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:37.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:27.873" grpc.time_ms=7.061 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 252.899526] gitaly[915]: time="2026-10-04T19:00:27.890Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:00:37.014" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:27.885" grpc.time_ms=5.209 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 252.934633] gitaly[915]: time="2026-10-04T19:00:27.942Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T19:00:37.060" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:27.931" grpc.time_ms=11.059 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764gitlab # [ 252.955970] nixbot[2691]: INFO: ::ffff:127.0.0.1:58866 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1765gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.05 seconds)1766gitlab # [ 252.982208] gitaly[915]: time="2026-10-04T19:00:27.990Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:27.989" grpc.time_ms=0.448 pid=915 span.kind=server system=grpc1767gitlab # [ 253.022773] gitaly[915]: time="2026-10-04T19:00:28.030Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:28.030" grpc.time_ms=0.682 pid=915 span.kind=server system=grpc1768gitlab # [ 253.040616] gitaly[915]: time="2026-10-04T19:00:28.048Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 diskcache=7de4ea67-5932-4102-9cdb-7a21135c385b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:37.098" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:27.969" pid=915 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1769gitlab # [ 253.050048] gitaly[915]: time="2026-10-04T19:00:28.058Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5090 command.oublock=8 command.real_time_ms=72 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:37.098" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:27.969" grpc.time_ms=79.513 pid=915 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1770gitlab # [ 253.063299] gitaly[915]: time="2026-10-04T19:00:28.071Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:00:38.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:28.071" grpc.time_ms=0.21 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1771gitlab # [ 253.081785] gitaly[915]: time="2026-10-04T19:00:28.089Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=943 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T19:00:38.005" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:28.075" grpc.time_ms=13.98 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772gitlab # [ 253.158215] gitaly[915]: time="2026-10-04T19:00:28.166Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-04T19:00:58.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:28.157" grpc.time_ms=8.143 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 253.216236] gitaly[915]: time="2026-10-04T19:00:28.224Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T19:00:38.152" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:28.223" grpc.time_ms=0.437 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774gitlab # [ 253.225544] gitaly[915]: time="2026-10-04T19:00:28.233Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T19:00:38.155" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T19:00:28.225" grpc.time_ms=7.569 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 253.254808] gitaly[915]: time="2026-10-04T19:00:28.262Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:38.182" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:28.252" grpc.time_ms=9.811 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1776gitlab # [ 253.272515] gitaly[915]: time="2026-10-04T19:00:28.280Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:38.200" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:28.270" grpc.time_ms=9.494 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1777gitlab # [ 253.304423] gitaly[915]: time="2026-10-04T19:00:28.312Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T19:00:58.197" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:28.297" grpc.time_ms=14.161 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1778gitlab # [ 253.669265] gitaly[915]: time="2026-10-04T19:00:28.677Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T19:00:38.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:28.675" grpc.time_ms=1.357 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 253.734872] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:28.741Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 66b3c13412ea8aafe07a6f1c588cbcbdda058456 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"795db3b9998a0cf97beae68e","created_at":"2026-10-04T18:59:40.284Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-b8058c408d504c04","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:59:40.292Z","job_size_bytes":185,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-795db3b9998a0cf97beae68e: done: 0.914257 sec","job_status":"done","queue_duration_s":47.535213,"scheduling_latency_s":47.535227,"gitaly_calls":13,"gitaly_duration_s":0.233753,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.080302,"redis_read_bytes":2103,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.004528,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008685,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001569,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003196,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.020698,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.041626,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.181,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.203,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.532956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003481727,"exclusive_lock_hold_duration_s":0.032940221,"duration_s":0.914257,"completed_at":"2026-10-04T19:00:28.741Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095159,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1780gitlab # [ 253.803239] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:28.743Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"2834d5b68e6522e3367e86d6","created_at":"2026-10-04T18:59:45.680Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.690Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-2834d5b68e6522e3367e86d6: start","job_status":"start","queue_duration_s":43.052838,"scheduling_latency_s":43.052854}1781gitlab # [ 253.828223] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:28.835Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"2834d5b68e6522e3367e86d6","created_at":"2026-10-04T18:59:45.680Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.690Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-2834d5b68e6522e3367e86d6: done: 0.092499 sec","job_status":"done","queue_duration_s":43.052838,"scheduling_latency_s":43.052854,"redis_calls":16,"redis_duration_s":0.003814,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002108,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033468,"duration_s":0.092499,"completed_at":"2026-10-04T19:00:28.835Z","load_balancing_strategy":"primary","db_duration_s":0.049581,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1782gitlab # [ 253.829722] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:28.837Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2f34215c5e5b2fbac9c5fb62","created_at":"2026-10-04T18:59:45.691Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.694Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2f34215c5e5b2fbac9c5fb62: start","job_status":"start","queue_duration_s":43.142722,"scheduling_latency_s":43.142738}1783gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1784gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.04 seconds)1785gitlab # [ 253.970907] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:28.978Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2f34215c5e5b2fbac9c5fb62","created_at":"2026-10-04T18:59:45.691Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.694Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2f34215c5e5b2fbac9c5fb62: done: 0.141166 sec","job_status":"done","queue_duration_s":43.142722,"scheduling_latency_s":43.142738,"redis_calls":21,"redis_duration_s":0.004706,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001878,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000301714,"duration_s":0.141166,"completed_at":"2026-10-04T19:00:28.978Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026558,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786gitlab # [ 253.998192] nixbot[2691]: INFO: ::ffff:127.0.0.1:52390 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1787gitlab # [ 254.040321] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.048Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bf79351f2a775cc439a528bc","created_at":"2026-10-04T18:59:45.774Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.779Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bf79351f2a775cc439a528bc: start","job_status":"start","queue_duration_s":43.268397,"scheduling_latency_s":43.268431}1788gitlab # [ 254.203454] gitaly[915]: time="2026-10-04T19:00:29.210Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:29.210" grpc.time_ms=0.455 pid=915 span.kind=server system=grpc1789gitlab # [ 254.252114] gitaly[915]: time="2026-10-04T19:00:29.260Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:29.259" grpc.time_ms=0.511 pid=915 span.kind=server system=grpc1790gitlab # [ 254.308839] gitaly[915]: time="2026-10-04T19:00:29.316Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:29.315" grpc.time_ms=0.744 pid=915 span.kind=server system=grpc1791gitlab # [ 254.335957] gitaly[915]: time="2026-10-04T19:00:29.343Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 diskcache=dd5097b2-409d-4db7-bc4c-67eedecec6ea grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:00:59.068" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:29.169" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792gitlab # [ 254.336504] gitaly[915]: time="2026-10-04T19:00:29.344Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=158 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7551 command.oublock=0 command.real_time_ms=166 command.spawn_token_fork_ms=0 command.system_time_ms=111 command.user_time_ms=47 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:00:59.068" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:29.169" grpc.time_ms=174.357 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793gitlab # [ 254.432293] gitaly[915]: time="2026-10-04T19:00:29.440Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:00:59.338" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:29.438" grpc.time_ms=1.341 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1794gitlab # [ 254.462477] gitaly[915]: time="2026-10-04T19:00:29.470Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JMGN2411BGX0G8Q5AG71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:00:59.368" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:29.469" grpc.time_ms=1.005 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1795gitlab # [ 254.486041] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.479Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bf79351f2a775cc439a528bc","created_at":"2026-10-04T18:59:45.774Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.779Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2570,"job_status":"rescheduled","message":"PipelineProcessWorker JID-bf79351f2a775cc439a528bc: rescheduled"}1796gitlab # [ 254.486186] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.485Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bf79351f2a775cc439a528bc","created_at":"2026-10-04T18:59:45.774Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-97dfe91c26c34fd8","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:59:45.779Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bf79351f2a775cc439a528bc: done: 0.438099 sec","job_status":"done","queue_duration_s":43.268397,"scheduling_latency_s":43.268431,"gitaly_calls":3,"gitaly_duration_s":0.185411,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025711,"redis_read_bytes":460,"redis_write_bytes":29295,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001335,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001174,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009931,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.003859,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15582,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005065,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000582,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00065,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.022,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.149104,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.37617797799999997,"duration_s":0.438099,"completed_at":"2026-10-04T19:00:29.485Z","load_balancing_strategy":"primary","db_duration_s":0.070876,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1797gitlab # [ 254.528183] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.488Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","66b3c13412ea8aafe07a6f1c588cbcbdda058456 7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"df19720632e6a5bcf1262683","created_at":"2026-10-04T19:00:26.757Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-f445010bcde6455e","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:26.759Z","job_size_bytes":185,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-df19720632e6a5bcf1262683: start","job_status":"start","queue_duration_s":2.729053,"scheduling_latency_s":2.729109}1798gitlab # [ 254.528796] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.507Z","message":"changes: \"66b3c13412ea8aafe07a6f1c588cbcbdda058456 7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 refs/heads/master\\n\""}1799gitlab # [ 254.537553] gitaly[915]: time="2026-10-04T19:00:29.545Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:39.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:29.533" grpc.time_ms=11.509 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 254.586910] gitaly[915]: time="2026-10-04T19:00:29.594Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:00:39.053" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:29.584" grpc.time_ms=10.524 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 254.599389] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.600Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ea4f6f1ce9c974b78393961a","created_at":"2026-10-04T19:00:29.597Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"288f5cda13b7293a742cd39a","job_size_bytes":46,"pid":2570,"job_status":"deduplicated","message":"ProjectCacheWorker JID-ea4f6f1ce9c974b78393961a: deduplicated: until executing","deduplication.type":"until executing"}1802gitlab # [ 254.626936] gitaly[915]: time="2026-10-04T19:00:29.634Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T19:00:39.093" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:29.624" grpc.time_ms=10.749 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab # [ 254.646957] gitaly[915]: time="2026-10-04T19:00:29.654Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-04T19:00:59.541" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:29.642" grpc.time_ms=12.589 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 254.665192] gitaly[915]: time="2026-10-04T19:00:29.673Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T19:00:39.136" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T19:00:29.666" grpc.time_ms=5.989 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 254.683117] gitaly[915]: time="2026-10-04T19:00:29.691Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:39.011" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:29.682" grpc.time_ms=8.857 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 254.794541] gitaly[915]: time="2026-10-04T19:00:29.802Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T19:00:39.130" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:29.801" grpc.time_ms=1.027 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 254.853156] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.859Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","66b3c13412ea8aafe07a6f1c588cbcbdda058456 7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"df19720632e6a5bcf1262683","created_at":"2026-10-04T19:00:26.757Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-f445010bcde6455e","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:26.759Z","job_size_bytes":185,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-df19720632e6a5bcf1262683: done: 0.37187 sec","job_status":"done","queue_duration_s":2.729053,"scheduling_latency_s":2.729109,"gitaly_calls":7,"gitaly_duration_s":0.153413,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.035342,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.003426,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001335,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004801,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.017148,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008632,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.028,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.167845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000974146,"exclusive_lock_hold_duration_s":0.0577931,"duration_s":0.37187,"completed_at":"2026-10-04T19:00:29.859Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028018,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1808gitlab # [ 254.917324] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:29.924Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"288f5cda13b7293a742cd39a","created_at":"2026-10-04T19:00:27.892Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:27.909Z","job_size_bytes":46,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-288f5cda13b7293a742cd39a: start","job_status":"start","queue_duration_s":2.015465,"scheduling_latency_s":2.015514}1809gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1810gitlab # [ 254.994643] gitaly[915]: time="2026-10-04T19:00:30.002Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-05T01:00:29.997" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:29.998" grpc.time_ms=4.178 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 255.043795] nixbot[2691]: INFO: ::ffff:127.0.0.1:52396 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1812gitlab # [ 255.044717] gitaly[915]: time="2026-10-04T19:00:30.050Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T19:01:00.035" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.036" grpc.time_ms=13.325 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.07 seconds)1814gitlab # [ 255.088264] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:30.095Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"288f5cda13b7293a742cd39a","created_at":"2026-10-04T19:00:27.892Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:27.909Z","job_size_bytes":46,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-288f5cda13b7293a742cd39a: done: 0.171024 sec","job_status":"done","queue_duration_s":2.015465,"scheduling_latency_s":2.015514,"gitaly_calls":2,"gitaly_duration_s":0.024095,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.044043,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001612,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.012824,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001761,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.027089,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000405638,"duration_s":0.171024,"completed_at":"2026-10-04T19:00:30.095Z","load_balancing_strategy":"primary","db_duration_s":0.015188,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1815gitlab # [ 255.142127] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:30.149Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"56f23af971f43406daffaf30","created_at":"2026-10-04T19:00:27.923Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:27.927Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-56f23af971f43406daffaf30: start","job_status":"start","queue_duration_s":2.222319,"scheduling_latency_s":2.22236}1816gitlab # [ 255.208306] gitaly[915]: time="2026-10-04T19:00:30.215Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 diskcache=f1ff6557-6640-4969-b8c6-4baf711f1252 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:00:30.183" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.183" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 255.208862] gitaly[915]: time="2026-10-04T19:00:30.216Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1287 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:00:30.183" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.183" grpc.time_ms=32.123 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 255.289272] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:30.296Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"56f23af971f43406daffaf30","created_at":"2026-10-04T19:00:27.923Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:27.927Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-56f23af971f43406daffaf30: done: 0.146458 sec","job_status":"done","queue_duration_s":2.222319,"scheduling_latency_s":2.22236,"gitaly_calls":1,"gitaly_duration_s":0.058406,"redis_calls":15,"redis_duration_s":0.005021,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000641,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000921,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.027,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.110208421,"duration_s":0.146458,"completed_at":"2026-10-04T19:00:30.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1819gitlab # [ 255.359770] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:30.367Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d2b061f66c60e926ff5d0caa","created_at":"2026-10-04T19:00:28.066Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:28.068Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d2b061f66c60e926ff5d0caa: start","job_status":"start","queue_duration_s":2.298323,"scheduling_latency_s":2.298373}1820gitlab # [ 255.411549] gitaly[915]: time="2026-10-04T19:00:30.419Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T19:01:00.409" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.409" grpc.time_ms=9.336 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 255.427924] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:30.435Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d2b061f66c60e926ff5d0caa","created_at":"2026-10-04T19:00:28.066Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:28.068Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d2b061f66c60e926ff5d0caa: done: 0.0684 sec","job_status":"done","queue_duration_s":2.298323,"scheduling_latency_s":2.298373,"gitaly_calls":1,"gitaly_duration_s":0.012904,"redis_calls":16,"redis_duration_s":0.006376,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000376,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002702,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0684,"completed_at":"2026-10-04T19:00:30.435Z","load_balancing_strategy":"primary","db_duration_s":0.004216,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1822gitlab # [ 255.470665] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:30.478Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"36d59580300d6ff5df86c47a","created_at":"2026-10-04T19:00:28.556Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fc01d6f69eb28dc74f1fa0eb953a8b86e5712dea057c7e74e87223693b3a78c","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:28.560Z","job_size_bytes":467,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-36d59580300d6ff5df86c47a: start","job_status":"start","queue_duration_s":1.917116,"scheduling_latency_s":1.917164}1823gitlab # [ 255.525899] gitaly[915]: time="2026-10-04T19:00:30.533Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:00.531" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.532" grpc.time_ms=1.011 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 255.549104] gitaly[915]: time="2026-10-04T19:00:30.557Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:00:40.015" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:30.545" grpc.time_ms=11.081 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825gitlab # [ 255.675360] gitaly[915]: time="2026-10-04T19:00:30.683Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:01:00.681" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.681" grpc.time_ms=1.282 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1826gitlab # [ 255.713356] gitaly[915]: time="2026-10-04T19:00:30.721Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=33 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T19:01:00.716" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:30.716" grpc.time_ms=3.988 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827gitlab # [ 255.848094] gitaly[915]: time="2026-10-04T19:00:30.855Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:40.172" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:30.843" grpc.time_ms=12.342 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1829gitlab # [ 256.114707] nixbot[2691]: INFO: ::ffff:127.0.0.1:52404 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1830gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.05 seconds)1831gitlab # [ 256.217028] gitaly[915]: time="2026-10-04T19:00:31.224Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T19:01:01.205" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:31.206" grpc.time_ms=18.073 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 256.234289] gitaly[915]: time="2026-10-04T19:00:31.242Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444JEK6V14KGP481M807Z43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T19:01:01.229" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:31.229" grpc.time_ms=12.371 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833gitlab # [ 256.304830] sidekiq-cluster[2570]: {"severity":"WARN","time":"2026-10-04T19:00:31.311Z","class":"CreatePipelineWorker","correlation_id":"01M444JEK6V14KGP481M807Z43","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"66b3c13412ea8aafe07a6f1c588cbcbdda058456","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"66b3c13412ea8aafe07a6f1c588cbcbdda058456","ref":"refs/heads/master"},"retry":0}1834gitlab # [ 256.310496] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"36d59580300d6ff5df86c47a","created_at":"2026-10-04T19:00:28.556Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fc01d6f69eb28dc74f1fa0eb953a8b86e5712dea057c7e74e87223693b3a78c","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:28.560Z","job_size_bytes":467,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-36d59580300d6ff5df86c47a: done: 0.840183 sec","job_status":"done","queue_duration_s":1.917116,"scheduling_latency_s":1.917164,"gitaly_calls":7,"gitaly_duration_s":0.181102,"redis_calls":51,"redis_duration_s":0.012125,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001057,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004856,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003405,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.605642,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000376026,"duration_s":0.840183,"completed_at":"2026-10-04T19:00:31.318Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056516,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1835gitlab # [ 256.330517] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.319Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"7e707db4-2e43-431c-b392-435536a31e93\"}"],"class":"WebHookWorker","jid":"c10dc46680b2543c46251619","created_at":"2026-10-04T19:00:28.650Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f5ad1c1c56fdeaa033f019e32b08fa52c061b2a7ee7cfea1a2b70d9f8be6bce8","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:28.655Z","job_size_bytes":1751,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c10dc46680b2543c46251619: start","job_status":"start","queue_duration_s":2.663954,"scheduling_latency_s":2.66397}1836gitlab # [ 256.422106] nixbot[2691]: INFO: ::ffff:127.0.0.1:52410 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1837gitlab # [ 256.443123] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.450Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"7e707db4-2e43-431c-b392-435536a31e93\"}"],"class":"WebHookWorker","jid":"c10dc46680b2543c46251619","created_at":"2026-10-04T19:00:28.650Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-3629ea23940442c5","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f5ad1c1c56fdeaa033f019e32b08fa52c061b2a7ee7cfea1a2b70d9f8be6bce8","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:28.655Z","job_size_bytes":1751,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c10dc46680b2543c46251619: done: 0.130302 sec","job_status":"done","queue_duration_s":2.663954,"scheduling_latency_s":2.66397,"redis_calls":19,"redis_duration_s":0.003997,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00118,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000394,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00158,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.03747655099999747,"cpu_s":0.077756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000420724,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.130302,"completed_at":"2026-10-04T19:00:31.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1838gitlab # [ 256.447937] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.452Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1d92facdbc0e290468822884","created_at":"2026-10-04T19:00:29.157Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.160Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1d92facdbc0e290468822884: start","job_status":"start","queue_duration_s":2.291922,"scheduling_latency_s":2.291938}1839gitlab # [ 256.498680] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.506Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1d92facdbc0e290468822884","created_at":"2026-10-04T19:00:29.157Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.160Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1d92facdbc0e290468822884: done: 0.053641 sec","job_status":"done","queue_duration_s":2.291922,"scheduling_latency_s":2.291938,"redis_calls":10,"redis_duration_s":0.001444,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212038,"duration_s":0.053641,"completed_at":"2026-10-04T19:00:31.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012114,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1840gitlab # [ 256.542643] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.511Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"871486d8fa59daf6399b1f28","created_at":"2026-10-04T19:00:29.352Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.357Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-871486d8fa59daf6399b1f28: start","job_status":"start","queue_duration_s":2.153707,"scheduling_latency_s":2.153722}1841gitlab # [ 256.543452] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.535Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"871486d8fa59daf6399b1f28","created_at":"2026-10-04T19:00:29.352Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.357Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-871486d8fa59daf6399b1f28: done: 0.023738 sec","job_status":"done","queue_duration_s":2.153707,"scheduling_latency_s":2.153722,"redis_calls":10,"redis_duration_s":0.00223,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000382,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000347809,"duration_s":0.023738,"completed_at":"2026-10-04T19:00:31.534Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1842gitlab # [ 256.560509] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e5a2d8f2031d91a83fb83059","created_at":"2026-10-04T19:00:29.369Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.376Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e5a2d8f2031d91a83fb83059: start","job_status":"start","queue_duration_s":2.159766,"scheduling_latency_s":2.15978}1843gitlab # [ 256.603699] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.560Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e5a2d8f2031d91a83fb83059","created_at":"2026-10-04T19:00:29.369Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.376Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e5a2d8f2031d91a83fb83059: done: 0.024097 sec","job_status":"done","queue_duration_s":2.159766,"scheduling_latency_s":2.15978,"redis_calls":10,"redis_duration_s":0.002084,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000307581,"duration_s":0.024097,"completed_at":"2026-10-04T19:00:31.560Z","load_balancing_strategy":"primary","db_duration_s":0.005397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1844gitlab # [ 256.647656] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.561Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a16f2c0d571269ef2c23fdb9","created_at":"2026-10-04T19:00:29.388Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.392Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a16f2c0d571269ef2c23fdb9: start","job_status":"start","queue_duration_s":2.16945,"scheduling_latency_s":2.169463}1845gitlab # [ 256.648548] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.621Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a16f2c0d571269ef2c23fdb9","created_at":"2026-10-04T19:00:29.388Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.392Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a16f2c0d571269ef2c23fdb9: done: 0.059514 sec","job_status":"done","queue_duration_s":2.16945,"scheduling_latency_s":2.169463,"redis_calls":13,"redis_duration_s":0.002644,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000384,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000665,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270425,"duration_s":0.059514,"completed_at":"2026-10-04T19:00:31.620Z","load_balancing_strategy":"primary","db_duration_s":0.006029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1846gitlab # [ 256.690770] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.622Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"efe26bbab5c69d13832a428d","created_at":"2026-10-04T19:00:29.392Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.395Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-efe26bbab5c69d13832a428d: start","job_status":"start","queue_duration_s":2.226865,"scheduling_latency_s":2.226885}1847gitlab # [ 256.691604] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.646Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"efe26bbab5c69d13832a428d","created_at":"2026-10-04T19:00:29.392Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.395Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-efe26bbab5c69d13832a428d: done: 0.023781 sec","job_status":"done","queue_duration_s":2.226865,"scheduling_latency_s":2.226885,"redis_calls":10,"redis_duration_s":0.002202,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341943,"duration_s":0.023781,"completed_at":"2026-10-04T19:00:31.646Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003979,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1848gitlab # [ 256.734194] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.647Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f8b864b34ab0f2c0c731f945","created_at":"2026-10-04T19:00:29.396Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.399Z","job_size_bytes":19,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f8b864b34ab0f2c0c731f945: start","job_status":"start","queue_duration_s":2.248152,"scheduling_latency_s":2.248163}1849gitlab # [ 256.776524] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.669Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f8b864b34ab0f2c0c731f945","created_at":"2026-10-04T19:00:29.396Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.399Z","job_size_bytes":19,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f8b864b34ab0f2c0c731f945: done: 0.02198 sec","job_status":"done","queue_duration_s":2.248152,"scheduling_latency_s":2.248163,"redis_calls":11,"redis_duration_s":0.002349,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000372,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270146,"duration_s":0.02198,"completed_at":"2026-10-04T19:00:31.669Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005125,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1850gitlab # [ 256.817723] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.670Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"82f79e2aec90e84a2a384a5f","created_at":"2026-10-04T19:00:29.401Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.404Z","job_size_bytes":66,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-82f79e2aec90e84a2a384a5f: start","job_status":"start","queue_duration_s":2.266577,"scheduling_latency_s":2.266589}1851gitlab # [ 256.818644] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.720Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"82f79e2aec90e84a2a384a5f","created_at":"2026-10-04T19:00:29.401Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.404Z","job_size_bytes":66,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-82f79e2aec90e84a2a384a5f: done: 0.049831 sec","job_status":"done","queue_duration_s":2.266577,"scheduling_latency_s":2.266589,"redis_calls":10,"redis_duration_s":0.002225,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00054,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002816,"duration_s":0.049831,"completed_at":"2026-10-04T19:00:31.720Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1852gitlab # [ 256.864229] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.721Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"021c39c81891d455f43a2f69","created_at":"2026-10-04T19:00:29.404Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.407Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-021c39c81891d455f43a2f69: start","job_status":"start","queue_duration_s":2.314315,"scheduling_latency_s":2.314329}1853gitlab # [ 256.864965] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.748Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"021c39c81891d455f43a2f69","created_at":"2026-10-04T19:00:29.404Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.407Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-021c39c81891d455f43a2f69: done: 0.026605 sec","job_status":"done","queue_duration_s":2.314315,"scheduling_latency_s":2.314329,"redis_calls":10,"redis_duration_s":0.001823,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000361,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000263162,"duration_s":0.026605,"completed_at":"2026-10-04T19:00:31.748Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1854gitlab # [ 256.914410] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8cfbe9e5f5832cb76d636ec5","created_at":"2026-10-04T19:00:29.412Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.414Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8cfbe9e5f5832cb76d636ec5: start","job_status":"start","queue_duration_s":2.334608,"scheduling_latency_s":2.334621}1855gitlab # [ 256.914996] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.775Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"8cfbe9e5f5832cb76d636ec5","created_at":"2026-10-04T19:00:29.412Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.414Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-8cfbe9e5f5832cb76d636ec5: done: 0.025746 sec","job_status":"done","queue_duration_s":2.334608,"scheduling_latency_s":2.334621,"redis_calls":10,"redis_duration_s":0.0017950000000000002,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000255061,"duration_s":0.025746,"completed_at":"2026-10-04T19:00:31.775Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1856gitlab # [ 256.955383] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.776Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f1e884e57f2e8c29d183b99a","created_at":"2026-10-04T19:00:29.415Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.417Z","job_size_bytes":6,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f1e884e57f2e8c29d183b99a: start","job_status":"start","queue_duration_s":2.358478,"scheduling_latency_s":2.358492}1857gitlab # [ 256.961967] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f1e884e57f2e8c29d183b99a","created_at":"2026-10-04T19:00:29.415Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.417Z","job_size_bytes":6,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f1e884e57f2e8c29d183b99a: done: 0.015374 sec","job_status":"done","queue_duration_s":2.358478,"scheduling_latency_s":2.358492,"redis_calls":10,"redis_duration_s":0.001881,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243047,"duration_s":0.015374,"completed_at":"2026-10-04T19:00:31.791Z","load_balancing_strategy":"primary","db_duration_s":0.002078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1858gitlab # [ 257.009229] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1ddf0af7630624083f38eb37","created_at":"2026-10-04T19:00:29.418Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.420Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1ddf0af7630624083f38eb37: start","job_status":"start","queue_duration_s":2.371917,"scheduling_latency_s":2.371931}1859gitlab # [ 257.016836] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.889Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1ddf0af7630624083f38eb37","created_at":"2026-10-04T19:00:29.418Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.420Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1ddf0af7630624083f38eb37: done: 0.096214 sec","job_status":"done","queue_duration_s":2.371917,"scheduling_latency_s":2.371931,"redis_calls":10,"redis_duration_s":0.0025770000000000003,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.069947438,"duration_s":0.096214,"completed_at":"2026-10-04T19:00:31.889Z","load_balancing_strategy":"primary","db_duration_s":0.005908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1860gitlab # [ 257.069098] gitaly[915]: time="2026-10-04T19:00:31.977Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC diskcache=c58a7042-ea17-4303-9c7c-11305afc61ba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:00:31.966" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:31.966" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861gitlab # [ 257.069699] gitaly[915]: time="2026-10-04T19:00:31.977Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:00:31.966" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:31.966" grpc.time_ms=10.55 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1863gitlab # [ 257.115718] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.890Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a86ff5f59de53c0fbc582991","created_at":"2026-10-04T19:00:29.426Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.429Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a86ff5f59de53c0fbc582991: start","job_status":"start","queue_duration_s":2.461477,"scheduling_latency_s":2.461497}1864gitlab # [ 257.116561] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.935Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a86ff5f59de53c0fbc582991","created_at":"2026-10-04T19:00:29.426Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.429Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a86ff5f59de53c0fbc582991: done: 0.044965 sec","job_status":"done","queue_duration_s":2.461477,"scheduling_latency_s":2.461497,"redis_calls":14,"redis_duration_s":0.002606,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001449,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000223771,"duration_s":0.044965,"completed_at":"2026-10-04T19:00:31.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865gitlab # [ 257.163760] gitaly[915]: time="2026-10-04T19:00:32.035Z" level=info msg="finished unary call with code OK" catfile.duration_ms=30 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=30 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:02.004" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.005" grpc.time_ms=30.752 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 257.164583] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.936Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f86bb0f27d46b920130e2f26","created_at":"2026-10-04T19:00:29.479Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.480Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f86bb0f27d46b920130e2f26: start","job_status":"start","queue_duration_s":2.455861,"scheduling_latency_s":2.455876}1867gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.10 seconds)1868gitlab # [ 257.165175] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.952Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f86bb0f27d46b920130e2f26","created_at":"2026-10-04T19:00:29.479Z","trace_propagation_headers":{"sentry-trace":"2629612d551a44868c72fcb747679141-7f9941153b0f4167","baggage":"sentry-trace_id=2629612d551a44868c72fcb747679141,sentry-release=Unknown"},"correlation_id":"01M444JMGN2411BGX0G8Q5AG71","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.480Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f86bb0f27d46b920130e2f26: done: 0.016409 sec","job_status":"done","queue_duration_s":2.455861,"scheduling_latency_s":2.455876,"redis_calls":8,"redis_duration_s":0.001197,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000238,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016409,"completed_at":"2026-10-04T19:00:31.952Z","load_balancing_strategy":"primary","db_duration_s":0.002211,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1869gitlab # [ 257.211756] gitaly[915]: time="2026-10-04T19:00:32.044Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:00:42.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.038" grpc.time_ms=6.233 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870gitlab # [ 257.212553] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.954Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b955d57ad40ec4fe7c7e9b72","created_at":"2026-10-04T19:00:29.607Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"56f23af971f43406daffaf30","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.610Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b955d57ad40ec4fe7c7e9b72: start","job_status":"start","queue_duration_s":2.343335,"scheduling_latency_s":2.343349}1871gitlab # [ 257.255343] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.988Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b955d57ad40ec4fe7c7e9b72","created_at":"2026-10-04T19:00:29.607Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"56f23af971f43406daffaf30","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.610Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b955d57ad40ec4fe7c7e9b72: done: 0.034261 sec","job_status":"done","queue_duration_s":2.343335,"scheduling_latency_s":2.343349,"gitaly_calls":1,"gitaly_duration_s":0.012882,"redis_calls":12,"redis_duration_s":0.002196,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000507,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000826,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.022520739,"duration_s":0.034261,"completed_at":"2026-10-04T19:00:31.988Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004153,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1872gitlab # [ 257.264873] gitaly[915]: time="2026-10-04T19:00:32.069Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:01:02.068" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.068" grpc.time_ms=0.945 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 257.274758] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 8151874gitlab # [ 257.276239] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:31.989Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2f4434a39faa311c1a3854bb","created_at":"2026-10-04T19:00:29.764Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1d96f1d03e2e3baf26b1cef1f00d9e90cb310846267090db110ad67780aed2ce","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.767Z","job_size_bytes":467,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2f4434a39faa311c1a3854bb: start","job_status":"start","queue_duration_s":2.222194,"scheduling_latency_s":2.222208}1875gitlab # [ 257.322857] nixbot[2691]: INFO: ::ffff:127.0.0.1:52422 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1876gitlab # [ 257.323756] gitaly[915]: time="2026-10-04T19:00:32.073Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=21 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T19:01:02.071" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.071" grpc.time_ms=1.973 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 257.324475] gitaly[915]: time="2026-10-04T19:00:32.106Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:42.067" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.097" grpc.time_ms=8.798 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 257.325603] gitaly[915]: time="2026-10-04T19:00:32.265Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444M1JTPF1X56ECNZXST33H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.258" grpc.time_ms=6.482 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1879gitlab # [ 257.334118] gitaly[915]: time="2026-10-04T19:00:32.337Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444M2CB9NEJ6W3N7RFQ6ATF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.326" pid=915 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1880gitlab # [ 257.343503] gitlab-workhorse[826]: time="2026-10-04T19:00:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671881gitlab # [ 257.345364] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 701882gitlab # [ 257.347082] gitaly[915]: time="2026-10-04T19:00:32.337Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2CB9NEJ6W3N7RFQ6ATF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.326" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1883gitlab # [ 257.396346] gitaly[915]: time="2026-10-04T19:00:32.337Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2CB9NEJ6W3N7RFQ6ATF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.326" grpc.time_ms=10.798 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1884gitlab # [ 257.396586] gitlab-workhorse[826]: time="2026-10-04T19:00:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3431885gitlab # [ 257.397078] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 343 "" "git/2.55.0" 481886gitlab # [ 257.397572] gitaly[915]: time="2026-10-04T19:00:32.389Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444M2EPWKMTA7TMEC69PP8Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.375" pid=915 remote_ip="2001:db8:1::1" request_sha=7baac7dd171494a207c59f2503e37fef552cd447 response_bytes=343 span.kind=server system=grpc1887gitlab # [ 257.398108] gitaly[915]: time="2026-10-04T19:00:32.389Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1195 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2EPWKMTA7TMEC69PP8Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.375" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1888gitlab # [ 257.398740] gitaly[915]: time="2026-10-04T19:00:32.389Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1195 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2EPWKMTA7TMEC69PP8Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.375" grpc.time_ms=14.193 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1889gitlab # [ 257.468538] gitaly[915]: time="2026-10-04T19:00:32.476Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T19:01:02.465" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.466" grpc.time_ms=10.352 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 257.478659] nixbot[2691]: {"ts": "2026-10-04T19:00:32.486Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1891gitlab # [ 257.496247] gitaly[915]: time="2026-10-04T19:00:32.504Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444KWF7WKSJSNAN1WEK0STC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T19:01:02.495" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.495" grpc.time_ms=8.031 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 257.510601] sidekiq-cluster[2570]: {"severity":"WARN","time":"2026-10-04T19:00:32.514Z","class":"CreatePipelineWorker","correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"66b3c13412ea8aafe07a6f1c588cbcbdda058456","after":"7edc24b17daf4bacdaa734228bc79d3da0d4e6e4","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"7edc24b17daf4bacdaa734228bc79d3da0d4e6e4","ref":"refs/heads/master"},"retry":0}1893gitlab # [ 257.512213] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.519Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2f4434a39faa311c1a3854bb","created_at":"2026-10-04T19:00:29.764Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1d96f1d03e2e3baf26b1cef1f00d9e90cb310846267090db110ad67780aed2ce","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.767Z","job_size_bytes":467,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2f4434a39faa311c1a3854bb: done: 0.530438 sec","job_status":"done","queue_duration_s":2.222194,"scheduling_latency_s":2.222208,"gitaly_calls":7,"gitaly_duration_s":0.097816,"redis_calls":33,"redis_duration_s":0.004678,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000517,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002389,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.076,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.34885,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.530438,"completed_at":"2026-10-04T19:00:32.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.078071,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1894gitlab # [ 257.553147] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.522Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6ac0ad0a-c746-46a3-9777-64a73031710a\"}"],"class":"WebHookWorker","jid":"d174ede2cd8eef0f19c08961","created_at":"2026-10-04T19:00:29.785Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f41147172d3ed3b1d620ddd2f6f9c73406fee2ce42daaf53af2d11514784bed9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.788Z","job_size_bytes":1749,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d174ede2cd8eef0f19c08961: start","job_status":"start","queue_duration_s":2.734544,"scheduling_latency_s":2.734569}1895gitlab # [ 257.554039] gitaly[915]: time="2026-10-04T19:00:32.544Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:02.542" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.543" grpc.time_ms=0.496 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1896gitlab # [ 257.589837] gitaly[915]: time="2026-10-04T19:00:32.597Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:42.171" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.591" grpc.time_ms=5.948 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 257.602587] nixbot[2691]: INFO: ::ffff:127.0.0.1:52436 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1898gitlab # [ 257.605883] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.613Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6ac0ad0a-c746-46a3-9777-64a73031710a\"}"],"class":"WebHookWorker","jid":"d174ede2cd8eef0f19c08961","created_at":"2026-10-04T19:00:29.785Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-4d0829dc06264537","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f41147172d3ed3b1d620ddd2f6f9c73406fee2ce42daaf53af2d11514784bed9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:29.788Z","job_size_bytes":1749,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d174ede2cd8eef0f19c08961: done: 0.091087 sec","job_status":"done","queue_duration_s":2.734544,"scheduling_latency_s":2.734569,"redis_calls":14,"redis_duration_s":0.001784,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000267,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000579,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007559340999989672,"cpu_s":0.018552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.091087,"completed_at":"2026-10-04T19:00:32.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1899gitlab # [ 257.608512] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.614Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3e71d8555ad32cbf3003b3f8","created_at":"2026-10-04T19:00:30.063Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-1d47f1c50e3a4b38","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:30.069Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3e71d8555ad32cbf3003b3f8: start","job_status":"start","queue_duration_s":2.54582,"scheduling_latency_s":2.545842}1900gitlab # [ 257.658823] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.652Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3e71d8555ad32cbf3003b3f8","created_at":"2026-10-04T19:00:30.063Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-1d47f1c50e3a4b38","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:30.069Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3e71d8555ad32cbf3003b3f8: done: 0.037508 sec","job_status":"done","queue_duration_s":2.54582,"scheduling_latency_s":2.545842,"redis_calls":19,"redis_duration_s":0.005105,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00288,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000412,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001379,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007138338,"duration_s":0.037508,"completed_at":"2026-10-04T19:00:32.652Z","load_balancing_strategy":"primary","db_duration_s":0.006092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1901gitlab # [ 257.664419] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.653Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c10dc46680b2543c46251619"],"class":"WebHooks::LogExecutionWorker","jid":"a5af32e31bfc5d127e21804f","created_at":"2026-10-04T19:00:31.433Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-fe57bb8d02704455","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:16f304a3febf8b2a40436359c6701c5c4853b7ababda6e801fda4131091e0784","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:31.441Z","job_size_bytes":2593,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a5af32e31bfc5d127e21804f: start","job_status":"start","queue_duration_s":1.212202,"scheduling_latency_s":1.212217}1902gitlab # [ 257.715265] nixbot[2691]: {"ts": "2026-10-04T19:00:32.642Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1903gitlab # [ 257.715962] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1391904gitlab # [ 257.716603] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.691Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c10dc46680b2543c46251619"],"class":"WebHooks::LogExecutionWorker","jid":"a5af32e31bfc5d127e21804f","created_at":"2026-10-04T19:00:31.433Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-fe57bb8d02704455","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:16f304a3febf8b2a40436359c6701c5c4853b7ababda6e801fda4131091e0784","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:31.441Z","job_size_bytes":2593,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a5af32e31bfc5d127e21804f: done: 0.037416 sec","job_status":"done","queue_duration_s":1.212202,"scheduling_latency_s":1.212217,"redis_calls":10,"redis_duration_s":0.001405,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267911,"duration_s":0.037416,"completed_at":"2026-10-04T19:00:32.690Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1905gitlab # [ 257.758505] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.692Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d174ede2cd8eef0f19c08961"],"class":"WebHooks::LogExecutionWorker","jid":"5ea203d3a51394f8e33c6499","created_at":"2026-10-04T19:00:32.605Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-0805fe9fcdfd41cc","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3659423f5363c4d17d9b01116d956e49b4003fb5e375848eeaf97686599e9ae1","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.608Z","job_size_bytes":2591,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5ea203d3a51394f8e33c6499: start","job_status":"start","queue_duration_s":0.083732,"scheduling_latency_s":0.083747}1906gitlab # [ 257.759219] gitaly[915]: time="2026-10-04T19:00:32.690Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2RCZ6BDZYGNPQFTDCMG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:02.688" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.689" grpc.time_ms=0.575 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 257.759976] nixbot[2691]: {"ts": "2026-10-04T19:00:32.754Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1908gitlab # [ 257.760772] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.711Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d174ede2cd8eef0f19c08961"],"class":"WebHooks::LogExecutionWorker","jid":"5ea203d3a51394f8e33c6499","created_at":"2026-10-04T19:00:32.605Z","trace_propagation_headers":{"sentry-trace":"562d9fb6880e48bfb297c58592005fef-0805fe9fcdfd41cc","baggage":"sentry-trace_id=562d9fb6880e48bfb297c58592005fef,sentry-release=Unknown"},"correlation_id":"01M444KWF7WKSJSNAN1WEK0STC","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3659423f5363c4d17d9b01116d956e49b4003fb5e375848eeaf97686599e9ae1","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.608Z","job_size_bytes":2591,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5ea203d3a51394f8e33c6499: done: 0.019822 sec","job_status":"done","queue_duration_s":0.083732,"scheduling_latency_s":0.083747,"redis_calls":7,"redis_duration_s":0.000956,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019822,"completed_at":"2026-10-04T19:00:32.711Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1909gitlab # [ 257.811736] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:32.713Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"046961ddcc92eaaec41a502c","created_at":"2026-10-04T19:00:32.626Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-2de85f3581414fdf","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.630Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-046961ddcc92eaaec41a502c: start","job_status":"start","queue_duration_s":0.083458,"scheduling_latency_s":0.083473}1910gitlab # [ 257.812661] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 991911gitlab # [ 257.813110] gitaly[915]: time="2026-10-04T19:00:32.716Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M2RCZ6BDZYGNPQFTDCMG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:42.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.711" grpc.time_ms=5.378 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 257.858033] gitaly[915]: time="2026-10-04T19:00:32.797Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2VPT1WM6AJBQJ9PG1GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:02.795" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.796" grpc.time_ms=1.645 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 257.858701] gitaly[915]: time="2026-10-04T19:00:32.864Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:32.864" grpc.time_ms=0.48 pid=915 span.kind=server system=grpc1914gitlab # [ 257.884353] gitaly[915]: time="2026-10-04T19:00:32.892Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:32.891" grpc.time_ms=0.452 pid=915 span.kind=server system=grpc1915gitlab # [ 257.886785] gitaly[915]: time="2026-10-04T19:00:32.894Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444M2VPT1WM6AJBQJ9PG1GS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:42.176" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.887" grpc.time_ms=7.486 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 257.925912] gitaly[915]: time="2026-10-04T19:00:32.933Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:32.933" grpc.time_ms=0.323 pid=915 span.kind=server system=grpc1917gitlab # [ 257.926370] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1741918gitlab # [ 257.928292] nixbot[2691]: {"ts": "2026-10-04T19:00:32.936Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/66b3c13412ea8aafe07a6f1c588cbcbdda058456 \"HTTP/1.1 201 Created\""}1919gitlab # [ 257.941562] gitaly[915]: time="2026-10-04T19:00:32.949Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444M3061DRF7XDRS1CWXXRV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.943" grpc.time_ms=6.45 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1920gitlab # [ 257.954236] gitaly[915]: time="2026-10-04T19:00:32.956Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF diskcache=362ad5f7-4b86-4d76-a4ee-a3b9cf0210c5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:01:02.763" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.764" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 257.963588] gitaly[915]: time="2026-10-04T19:00:32.956Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=59 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7276 command.oublock=0 command.real_time_ms=185 command.spawn_token_fork_ms=0 command.system_time_ms=26 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:01:02.763" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:32.764" grpc.time_ms=192.263 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 257.976234] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 471923gitlab # [ 257.991564] gitaly[915]: time="2026-10-04T19:00:32.999Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:01:02.098" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:32.998" grpc.time_ms=0.582 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 258.006395] gitaly[915]: time="2026-10-04T19:00:33.014Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444M31Q2VKQKSKR8ED20WXH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.996" pid=915 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1925gitlab # [ 258.006768] gitaly[915]: time="2026-10-04T19:00:33.014Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444M31Q2VKQKSKR8ED20WXH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.996" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1926gitlab # [ 258.007553] gitaly[915]: time="2026-10-04T19:00:33.014Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444M31Q2VKQKSKR8ED20WXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:32.996" grpc.time_ms=18.331 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1927gitlab # [ 258.009101] gitlab-workhorse[826]: time="2026-10-04T19:00:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671928gitlab # [ 258.009533] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:33 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 641929gitlab # [ 258.068739] gitaly[915]: time="2026-10-04T19:00:33.076Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:03.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:33.075" grpc.time_ms=0.589 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 258.077908] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.085Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"cd3badb755ba61e0ead5277a","created_at":"2026-10-04T19:00:33.084Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"046961ddcc92eaaec41a502c","job_size_bytes":3,"pid":2570,"job_status":"deduplicated","message":"PipelineProcessWorker JID-cd3badb755ba61e0ead5277a: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1931gitlab # [ 258.079746] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.087Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"046961ddcc92eaaec41a502c","created_at":"2026-10-04T19:00:32.626Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-2de85f3581414fdf","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.630Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2570,"job_status":"rescheduled","message":"PipelineProcessWorker JID-046961ddcc92eaaec41a502c: rescheduled"}1932gitlab # [ 258.085425] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.093Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"046961ddcc92eaaec41a502c","created_at":"2026-10-04T19:00:32.626Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-2de85f3581414fdf","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.630Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-046961ddcc92eaaec41a502c: done: 0.379817 sec","job_status":"done","queue_duration_s":0.083458,"scheduling_latency_s":0.083473,"gitaly_calls":3,"gitaly_duration_s":0.211433,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011646,"redis_read_bytes":227,"redis_write_bytes":18466,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000315,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000827,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003252,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.004768,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000142,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000778,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001249,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090915,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.348441491,"duration_s":0.379817,"completed_at":"2026-10-04T19:00:33.093Z","load_balancing_strategy":"primary","db_duration_s":0.064556,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1933gitlab # [ 258.136649] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.100Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6c2347dab3b3ef10320442f4","created_at":"2026-10-04T19:00:32.640Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-701bbf240917428c","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.646Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6c2347dab3b3ef10320442f4: start","job_status":"start","queue_duration_s":0.454396,"scheduling_latency_s":0.454413}1934gitlab # [ 258.137452] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:33 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 411935gitlab # [ 258.137956] gitaly[915]: time="2026-10-04T19:00:33.115Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M35J6K131BTGXSTZAMSE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.108" grpc.time_ms=6.755 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1936gitlab # [ 258.149323] gitaly[915]: time="2026-10-04T19:00:33.157Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444M36XAD7NM2VBTJK05XPZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.149" pid=915 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1937gitlab # [ 258.154288] gitaly[915]: time="2026-10-04T19:00:33.157Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444M36XAD7NM2VBTJK05XPZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.149" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1938gitlab # [ 258.206437] gitlab-workhorse[826]: time="2026-10-04T19:00:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671939gitlab # [ 258.207141] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:33 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 401940gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1941gitlab # [ 258.207672] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6c2347dab3b3ef10320442f4","created_at":"2026-10-04T19:00:32.640Z","trace_propagation_headers":{"sentry-trace":"4b5e4528728a4eb39f694596fe06241f-701bbf240917428c","baggage":"sentry-trace_id=4b5e4528728a4eb39f694596fe06241f,sentry-release=Unknown"},"correlation_id":"01M444JEK6V14KGP481M807Z43","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.646Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6c2347dab3b3ef10320442f4: done: 0.071934 sec","job_status":"done","queue_duration_s":0.454396,"scheduling_latency_s":0.454413,"redis_calls":11,"redis_duration_s":0.002549,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000298,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001774,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000298083,"duration_s":0.071934,"completed_at":"2026-10-04T19:00:33.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 258.213702] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.173Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9a8477d9b6184de7e3c24a94","created_at":"2026-10-04T19:00:32.755Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.758Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9a8477d9b6184de7e3c24a94: start","job_status":"start","queue_duration_s":0.415171,"scheduling_latency_s":0.415187}1943gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.04 seconds)1944gitlab # [ 258.228764] gitaly[915]: time="2026-10-04T19:00:33.157Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444M36XAD7NM2VBTJK05XPZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.149" grpc.time_ms=7.956 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1945gitlab # [ 258.241071] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9a8477d9b6184de7e3c24a94","created_at":"2026-10-04T19:00:32.755Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.758Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9a8477d9b6184de7e3c24a94: done: 0.071483 sec","job_status":"done","queue_duration_s":0.415171,"scheduling_latency_s":0.415187,"redis_calls":7,"redis_duration_s":0.004321,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.071483,"completed_at":"2026-10-04T19:00:33.244Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039577,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1946gitlab # [ 258.296728] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.247Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7d485c947d4a4a8b83e8976b","created_at":"2026-10-04T19:00:32.961Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.963Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7d485c947d4a4a8b83e8976b: start","job_status":"start","queue_duration_s":0.284634,"scheduling_latency_s":0.284657}1947gitlab # [ 258.297650] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.271Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7d485c947d4a4a8b83e8976b","created_at":"2026-10-04T19:00:32.961Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.963Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7d485c947d4a4a8b83e8976b: done: 0.023141 sec","job_status":"done","queue_duration_s":0.284634,"scheduling_latency_s":0.284657,"redis_calls":7,"redis_duration_s":0.0011099999999999999,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023141,"completed_at":"2026-10-04T19:00:33.271Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1948gitlab # [ 258.339082] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.272Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5c266217c6ae24dc2307015c","created_at":"2026-10-04T19:00:32.968Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.971Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5c266217c6ae24dc2307015c: start","job_status":"start","queue_duration_s":0.300413,"scheduling_latency_s":0.300429}1949gitlab # [ 258.379517] nixbot[2691]: INFO: ::ffff:127.0.0.1:52450 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK1950gitlab # [ 258.380141] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"5c266217c6ae24dc2307015c","created_at":"2026-10-04T19:00:32.968Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.971Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-5c266217c6ae24dc2307015c: done: 0.017481 sec","job_status":"done","queue_duration_s":0.300413,"scheduling_latency_s":0.300429,"redis_calls":7,"redis_duration_s":0.001082,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017481,"completed_at":"2026-10-04T19:00:33.289Z","load_balancing_strategy":"primary","db_duration_s":0.002907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951gitlab # [ 258.425158] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"63d19c0c02013b84887ebaff","created_at":"2026-10-04T19:00:32.978Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.979Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-63d19c0c02013b84887ebaff: start","job_status":"start","queue_duration_s":0.310896,"scheduling_latency_s":0.310911}1952gitlab # [ 258.426067] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.312Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"63d19c0c02013b84887ebaff","created_at":"2026-10-04T19:00:32.978Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.979Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-63d19c0c02013b84887ebaff: done: 0.02255 sec","job_status":"done","queue_duration_s":0.310896,"scheduling_latency_s":0.310911,"redis_calls":10,"redis_duration_s":0.001457,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00051,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02255,"completed_at":"2026-10-04T19:00:33.312Z","load_balancing_strategy":"primary","db_duration_s":0.003861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 258.467393] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.313Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2c9ad8ad9ba173d737faddeb","created_at":"2026-10-04T19:00:32.979Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.980Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2c9ad8ad9ba173d737faddeb: start","job_status":"start","queue_duration_s":0.332944,"scheduling_latency_s":0.33296}1954gitlab # [ 258.468444] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.329Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2c9ad8ad9ba173d737faddeb","created_at":"2026-10-04T19:00:32.979Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.980Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2c9ad8ad9ba173d737faddeb: done: 0.015535 sec","job_status":"done","queue_duration_s":0.332944,"scheduling_latency_s":0.33296,"redis_calls":7,"redis_duration_s":0.001121,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015535,"completed_at":"2026-10-04T19:00:33.329Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1955gitlab # [ 258.512546] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.330Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0375e3df86e83b9159056cec","created_at":"2026-10-04T19:00:32.981Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.982Z","job_size_bytes":19,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0375e3df86e83b9159056cec: start","job_status":"start","queue_duration_s":0.347615,"scheduling_latency_s":0.347629}1956gitlab # [ 258.514832] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0375e3df86e83b9159056cec","created_at":"2026-10-04T19:00:32.981Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.982Z","job_size_bytes":19,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0375e3df86e83b9159056cec: done: 0.014135 sec","job_status":"done","queue_duration_s":0.347615,"scheduling_latency_s":0.347629,"redis_calls":8,"redis_duration_s":0.001039,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014135,"completed_at":"2026-10-04T19:00:33.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1957gitlab # [ 258.559887] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.345Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1b979801abb4674edab38f19","created_at":"2026-10-04T19:00:32.983Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.986Z","job_size_bytes":66,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1b979801abb4674edab38f19: start","job_status":"start","queue_duration_s":0.358401,"scheduling_latency_s":0.358416}1958gitlab # [ 258.569366] gitaly[915]: time="2026-10-04T19:00:33.477Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:33.477" grpc.time_ms=0.436 pid=915 span.kind=server system=grpc1959gitlab # [ 258.577784] gitaly[915]: time="2026-10-04T19:00:33.497Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:33.496" grpc.time_ms=0.426 pid=915 span.kind=server system=grpc1960gitlab # [ 258.586887] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:33 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 3621961gitlab # [ 258.588526] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.358Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1b979801abb4674edab38f19","created_at":"2026-10-04T19:00:32.983Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.986Z","job_size_bytes":66,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1b979801abb4674edab38f19: done: 0.012812 sec","job_status":"done","queue_duration_s":0.358401,"scheduling_latency_s":0.358416,"redis_calls":7,"redis_duration_s":0.000894,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012812,"completed_at":"2026-10-04T19:00:33.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 258.628234] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.359Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9dbba9f25cd333f56729616d","created_at":"2026-10-04T19:00:32.987Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.988Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9dbba9f25cd333f56729616d: start","job_status":"start","queue_duration_s":0.370766,"scheduling_latency_s":0.370783}1963gitlab # [ 258.673346] gitaly[915]: time="2026-10-04T19:00:33.548Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444M392EMBYE8HF2C5SAHA0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.541" grpc.time_ms=7.034 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1964gitlab # [ 258.674107] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.388Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9dbba9f25cd333f56729616d","created_at":"2026-10-04T19:00:32.987Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.988Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9dbba9f25cd333f56729616d: done: 0.029227 sec","job_status":"done","queue_duration_s":0.370766,"scheduling_latency_s":0.370783,"redis_calls":7,"redis_duration_s":0.0009209999999999999,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029227,"completed_at":"2026-10-04T19:00:33.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 258.694720] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7c9441bc414a66f8f3811fbf","created_at":"2026-10-04T19:00:32.988Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.989Z","job_size_bytes":6,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7c9441bc414a66f8f3811fbf: start","job_status":"start","queue_duration_s":0.399678,"scheduling_latency_s":0.399693}1966gitlab # [ 258.737524] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.397Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7c9441bc414a66f8f3811fbf","created_at":"2026-10-04T19:00:32.988Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.989Z","job_size_bytes":6,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7c9441bc414a66f8f3811fbf: done: 0.008605 sec","job_status":"done","queue_duration_s":0.399678,"scheduling_latency_s":0.399693,"redis_calls":7,"redis_duration_s":0.000845,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008605,"completed_at":"2026-10-04T19:00:33.397Z","load_balancing_strategy":"primary","db_duration_s":0.000259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1967gitlab # [ 258.749850] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2c49452967473cd71d9e9e37","created_at":"2026-10-04T19:00:32.989Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.990Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2c49452967473cd71d9e9e37: start","job_status":"start","queue_duration_s":0.407989,"scheduling_latency_s":0.408002}1968gitlab # [ 258.800708] gitlab-workhorse[826]: time="2026-10-04T19:00:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161969gitlab # [ 258.801539] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:33 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1481970gitlab # [ 258.802477] gitaly[915]: time="2026-10-04T19:00:33.697Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3MEEVF23KWJDHRQYYZT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.674" pid=915 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1971gitlab # [ 258.803458] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.412Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2c49452967473cd71d9e9e37","created_at":"2026-10-04T19:00:32.989Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:32.990Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2c49452967473cd71d9e9e37: done: 0.013632 sec","job_status":"done","queue_duration_s":0.407989,"scheduling_latency_s":0.408002,"redis_calls":7,"redis_duration_s":0.0009109999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013632,"completed_at":"2026-10-04T19:00:33.412Z","load_balancing_strategy":"primary","db_duration_s":0.001957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1972gitlab # [ 258.845856] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.413Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e5d1323869154ba36deefcf5","created_at":"2026-10-04T19:00:33.088Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.089Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e5d1323869154ba36deefcf5: start","job_status":"start","queue_duration_s":0.323268,"scheduling_latency_s":0.323282}1973gitlab # [ 258.846804] gitaly[915]: time="2026-10-04T19:00:33.697Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3MEEVF23KWJDHRQYYZT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.674" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1974gitlab # [ 258.847479] gitaly[915]: time="2026-10-04T19:00:33.698Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3MEEVF23KWJDHRQYYZT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:00:33.674" grpc.time_ms=23.449 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc1975gitlab # [ 258.859183] gitaly[915]: time="2026-10-04T19:00:33.771Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:03.770" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:33.770" grpc.time_ms=0.658 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 258.909683] gitaly[915]: time="2026-10-04T19:00:33.785Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T19:01:03.778" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:33.779" grpc.time_ms=6.086 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab # [ 258.910518] gitaly[915]: time="2026-10-04T19:00:33.797Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:43.080" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:33.790" grpc.time_ms=6.434 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978gitlab # [ 258.911204] gitaly[915]: time="2026-10-04T19:00:33.815Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T19:00:43.105" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.815" grpc.time_ms=0.201 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979gitlab # [ 258.911899] gitaly[915]: time="2026-10-04T19:00:33.822Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:00:43.107" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:33.818" grpc.time_ms=4.726 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 258.956843] gitaly[915]: time="2026-10-04T19:00:33.851Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:33.850" grpc.time_ms=1.314 pid=915 span.kind=server system=grpc1981gitlab # [ 258.957752] gitaly[915]: time="2026-10-04T19:00:33.864Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:33.863" grpc.time_ms=0.386 pid=915 span.kind=server system=grpc1982gitlab # [ 258.958181] gitaly[915]: time="2026-10-04T19:00:33.876Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF diskcache=d783172b-a909-40c2-878a-31b1d0561cef grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:01:03.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:33.454" pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 258.958722] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.949Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e5d1323869154ba36deefcf5","created_at":"2026-10-04T19:00:33.088Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-06c59400167f491e","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.089Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e5d1323869154ba36deefcf5: done: 0.53681 sec","job_status":"done","queue_duration_s":0.323268,"scheduling_latency_s":0.323282,"gitaly_calls":3,"gitaly_duration_s":0.4315,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008258,"redis_read_bytes":141,"redis_write_bytes":16718,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000322,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000686,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001726,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11224,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.003581,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000145,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00038,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080593,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.517257361,"duration_s":0.53681,"completed_at":"2026-10-04T19:00:33.949Z","load_balancing_strategy":"primary","db_duration_s":0.021843,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1984gitlab # [ 259.009871] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.950Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3b2d5ae01684e0946fcbb0e8","created_at":"2026-10-04T19:00:33.449Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.450Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3b2d5ae01684e0946fcbb0e8: start","job_status":"start","queue_duration_s":0.500405,"scheduling_latency_s":0.50042}1985gitlab # [ 259.012688] gitaly[915]: time="2026-10-04T19:00:33.877Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=93 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=823296 command.minflt=7453 command.oublock=0 command.real_time_ms=415 command.spawn_token_fork_ms=0 command.system_time_ms=62 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:01:03.453" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:33.454" grpc.time_ms=422.433 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 259.024926] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.975Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3b2d5ae01684e0946fcbb0e8","created_at":"2026-10-04T19:00:33.449Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.450Z","job_size_bytes":23,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3b2d5ae01684e0946fcbb0e8: done: 0.024412 sec","job_status":"done","queue_duration_s":0.500405,"scheduling_latency_s":0.50042,"redis_calls":7,"redis_duration_s":0.001031,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024412,"completed_at":"2026-10-04T19:00:33.975Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00436,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1987gitlab # [ 259.066096] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.976Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"21d24a6830125404882b9ff7","created_at":"2026-10-04T19:00:33.881Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.884Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-21d24a6830125404882b9ff7: start","job_status":"start","queue_duration_s":0.092171,"scheduling_latency_s":0.092185}1988gitlab # [ 259.106210] gitaly[915]: time="2026-10-04T19:00:33.917Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:01:03.016" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:33.916" grpc.time_ms=0.953 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989gitlab # [ 259.106972] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"21d24a6830125404882b9ff7","created_at":"2026-10-04T19:00:33.881Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.884Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-21d24a6830125404882b9ff7: done: 0.020831 sec","job_status":"done","queue_duration_s":0.092171,"scheduling_latency_s":0.092185,"redis_calls":7,"redis_duration_s":0.001142,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000318,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020831,"completed_at":"2026-10-04T19:00:33.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1990gitlab # [ 259.130183] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:33.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"348fbed250ce2548d32eaf9b","created_at":"2026-10-04T19:00:33.889Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.893Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-348fbed250ce2548d32eaf9b: start","job_status":"start","queue_duration_s":0.105238,"scheduling_latency_s":0.105253}1991gitlab # [ 259.178413] gitaly[915]: time="2026-10-04T19:00:33.942Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M2KM8882K8MXW7TMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:03.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:33.941" grpc.time_ms=0.546 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1992gitlab # [ 259.179159] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"348fbed250ce2548d32eaf9b","created_at":"2026-10-04T19:00:33.889Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.893Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-348fbed250ce2548d32eaf9b: done: 0.016688 sec","job_status":"done","queue_duration_s":0.105238,"scheduling_latency_s":0.105253,"redis_calls":7,"redis_duration_s":0.0009369999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016688,"completed_at":"2026-10-04T19:00:34.014Z","load_balancing_strategy":"primary","db_duration_s":0.002644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1993gitlab # [ 259.223953] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.015Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"3db925b1e4f9bfc09a5addac","created_at":"2026-10-04T19:00:33.899Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.900Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-3db925b1e4f9bfc09a5addac: start","job_status":"start","queue_duration_s":0.115095,"scheduling_latency_s":0.115109}1994gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4'1995gitlab # [ 259.224738] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"3db925b1e4f9bfc09a5addac","created_at":"2026-10-04T19:00:33.899Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.900Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-3db925b1e4f9bfc09a5addac: done: 0.023708 sec","job_status":"done","queue_duration_s":0.115095,"scheduling_latency_s":0.115109,"redis_calls":10,"redis_duration_s":0.001625,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000576,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023708,"completed_at":"2026-10-04T19:00:34.039Z","load_balancing_strategy":"primary","db_duration_s":0.003511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996gitlab # [ 259.264043] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"462c13c097d18e3932a6d0d0","created_at":"2026-10-04T19:00:33.900Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.901Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-462c13c097d18e3932a6d0d0: start","job_status":"start","queue_duration_s":0.138459,"scheduling_latency_s":0.138475}1997gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', in 0.04 seconds)1998(finished: subtest: push delivers the webhook, in 51.00 seconds)1999subtest: eval and build post commit statuses2000gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses2001gitlab # [ 259.264786] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"462c13c097d18e3932a6d0d0","created_at":"2026-10-04T19:00:33.900Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.901Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-462c13c097d18e3932a6d0d0: done: 0.01602 sec","job_status":"done","queue_duration_s":0.138459,"scheduling_latency_s":0.138475,"redis_calls":7,"redis_duration_s":0.001012,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01602,"completed_at":"2026-10-04T19:00:34.056Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002gitlab # [ 259.318291] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.057Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ef3467fa9a3bfcd684a2d570","created_at":"2026-10-04T19:00:33.902Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.903Z","job_size_bytes":19,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ef3467fa9a3bfcd684a2d570: start","job_status":"start","queue_duration_s":0.153475,"scheduling_latency_s":0.153489}2003gitlab # [ 259.318991] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.071Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ef3467fa9a3bfcd684a2d570","created_at":"2026-10-04T19:00:33.902Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.903Z","job_size_bytes":19,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ef3467fa9a3bfcd684a2d570: done: 0.01408 sec","job_status":"done","queue_duration_s":0.153475,"scheduling_latency_s":0.153489,"redis_calls":8,"redis_duration_s":0.001108,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01408,"completed_at":"2026-10-04T19:00:34.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses, in 0.09 seconds)2005gitlab # [ 259.371033] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.072Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d8105d9bd3cc591aec60d7c7","created_at":"2026-10-04T19:00:33.904Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.905Z","job_size_bytes":66,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d8105d9bd3cc591aec60d7c7: start","job_status":"start","queue_duration_s":0.166843,"scheduling_latency_s":0.166857}2006gitlab # [ 259.371802] nixbot[2691]: INFO: ::ffff:127.0.0.1:52460 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 200 OK2007gitlab # [ 259.373820] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.085Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"d8105d9bd3cc591aec60d7c7","created_at":"2026-10-04T19:00:33.904Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.905Z","job_size_bytes":66,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-d8105d9bd3cc591aec60d7c7: done: 0.013071 sec","job_status":"done","queue_duration_s":0.166843,"scheduling_latency_s":0.166857,"redis_calls":7,"redis_duration_s":0.001021,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013071,"completed_at":"2026-10-04T19:00:34.085Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 259.424651] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.086Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"93bdf029fc6016fc1b901fc8","created_at":"2026-10-04T19:00:33.905Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.906Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-93bdf029fc6016fc1b901fc8: start","job_status":"start","queue_duration_s":0.179617,"scheduling_latency_s":0.17963}2009gitlab # [ 259.429146] gitaly[915]: time="2026-10-04T19:00:34.381Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M4D3VRCPCZP7RE05RMR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:04.079" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:34.380" grpc.time_ms=0.619 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2010gitlab # [ 259.472776] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 732011gitlab # [ 259.473649] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.102Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"93bdf029fc6016fc1b901fc8","created_at":"2026-10-04T19:00:33.905Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.906Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-93bdf029fc6016fc1b901fc8: done: 0.015956 sec","job_status":"done","queue_duration_s":0.179617,"scheduling_latency_s":0.17963,"redis_calls":7,"redis_duration_s":0.000916,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015956,"completed_at":"2026-10-04T19:00:34.102Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2012gitlab # [ 259.480686] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.103Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ef2c13fc142f477ef59cee55","created_at":"2026-10-04T19:00:33.906Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.907Z","job_size_bytes":6,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ef2c13fc142f477ef59cee55: start","job_status":"start","queue_duration_s":0.19529,"scheduling_latency_s":0.195302}2013gitlab # [ 259.525451] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.111Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ef2c13fc142f477ef59cee55","created_at":"2026-10-04T19:00:33.906Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.907Z","job_size_bytes":6,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ef2c13fc142f477ef59cee55: done: 0.008364 sec","job_status":"done","queue_duration_s":0.19529,"scheduling_latency_s":0.195302,"redis_calls":7,"redis_duration_s":0.0009299999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008364,"completed_at":"2026-10-04T19:00:34.111Z","load_balancing_strategy":"primary","db_duration_s":0.000293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 259.532800] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.112Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"5d6aafe71a722d835292c96c","created_at":"2026-10-04T19:00:33.908Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.908Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-5d6aafe71a722d835292c96c: start","job_status":"start","queue_duration_s":0.203562,"scheduling_latency_s":0.203574}2015gitlab # [ 259.580760] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.127Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"5d6aafe71a722d835292c96c","created_at":"2026-10-04T19:00:33.908Z","trace_propagation_headers":{"sentry-trace":"4ffe05a7e3a642d1a92de7f0b10165f5-60f009730d4347ab","baggage":"sentry-trace_id=4ffe05a7e3a642d1a92de7f0b10165f5,sentry-release=Unknown"},"correlation_id":"01M444M2KM8882K8MXW7TMYMWF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:33.908Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-5d6aafe71a722d835292c96c: done: 0.015052 sec","job_status":"done","queue_duration_s":0.203562,"scheduling_latency_s":0.203574,"redis_calls":7,"redis_duration_s":0.001446,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015052,"completed_at":"2026-10-04T19:00:34.127Z","load_balancing_strategy":"primary","db_duration_s":0.002605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016gitlab # [ 259.636872] gitaly[915]: time="2026-10-04T19:00:34.644Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:00:34.644" grpc.time_ms=0.471 pid=915 span.kind=server system=grpc2017gitlab # [ 259.647213] gitaly[915]: time="2026-10-04T19:00:34.655Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:43.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.824" operations="[repack_refs]" pid=915 ref=refs/keep-around/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 remote_ip="2001:db8:1::1" revision=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 span.kind=server system=grpc username=root2018gitlab # [ 259.658416] gitaly[915]: time="2026-10-04T19:00:34.656Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:43.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.824" pid=915 ref=refs/keep-around/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-04 18:59:32.425143257 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 span.kind=server system=grpc username=root2019gitlab # [ 259.664575] gitaly[915]: time="2026-10-04T19:00:34.672Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 diskcache=81845931-4b07-45e5-a81e-ac1b1c3e7cd6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:43.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.824" pid=915 ref=refs/keep-around/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 remote_ip="2001:db8:1::1" revision=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 span.kind=server system=grpc username=root2020gitlab # [ 259.665217] gitaly[915]: time="2026-10-04T19:00:34.672Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=100 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=5288 command.oublock=8 command.real_time_ms=829 command.spawn_token_fork_ms=0 command.system_time_ms=86 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:43.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.824" grpc.time_ms=848.412 pid=915 ref=refs/keep-around/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 remote_ip="2001:db8:1::1" revision=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 span.kind=server system=grpc username=root2021gitlab # [ 259.669759] gitaly[915]: time="2026-10-04T19:00:34.677Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:04.677" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:34.677" grpc.time_ms=0.332 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2022gitlab # [ 259.675216] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.682Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bf5f93ca6d000f4ca5ed39d5","created_at":"2026-10-04T19:00:34.679Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-c2a4f27f98df4e3e","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.681Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bf5f93ca6d000f4ca5ed39d5: start","job_status":"start","queue_duration_s":0.001369,"scheduling_latency_s":0.001404}2023gitlab # [ 259.716958] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.710Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bf5f93ca6d000f4ca5ed39d5","created_at":"2026-10-04T19:00:34.679Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-c2a4f27f98df4e3e","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.681Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bf5f93ca6d000f4ca5ed39d5: done: 0.02755 sec","job_status":"done","queue_duration_s":0.001369,"scheduling_latency_s":0.001404,"redis_calls":7,"redis_duration_s":0.001185,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02755,"completed_at":"2026-10-04T19:00:34.710Z","load_balancing_strategy":"primary","db_duration_s":0.004926,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2024gitlab # [ 259.721676] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.712Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c2d3525bc6588a8e3eb42046","created_at":"2026-10-04T19:00:34.682Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-c2a4f27f98df4e3e","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.686Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c2d3525bc6588a8e3eb42046: start","job_status":"start","queue_duration_s":0.02591,"scheduling_latency_s":0.025945}2025gitlab # [ 259.738390] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 10182026gitlab # [ 259.742430] nixbot[2691]: {"ts": "2026-10-04T19:00:34.748Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 \"HTTP/1.1 201 Created\""}2027gitlab # [ 259.749986] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.757Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c2d3525bc6588a8e3eb42046","created_at":"2026-10-04T19:00:34.682Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-c2a4f27f98df4e3e","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.686Z","job_size_bytes":65,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c2d3525bc6588a8e3eb42046: done: 0.04579 sec","job_status":"done","queue_duration_s":0.02591,"scheduling_latency_s":0.025945,"redis_calls":10,"redis_duration_s":0.001859,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04579,"completed_at":"2026-10-04T19:00:34.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 259.819474] nixbot[2691]: {"ts": "2026-10-04T19:00:34.758Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-7edc24b17daf-5"}2029gitlab # [ 259.820085] gitaly[915]: time="2026-10-04T19:00:34.810Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:01:04.808" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:34.809" grpc.time_ms=0.655 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2030gitlab # [ 259.820785] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.758Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"eadf220546a377a1bd6551cd","created_at":"2026-10-04T19:00:34.732Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-c2a4f27f98df4e3e","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.735Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-eadf220546a377a1bd6551cd: start","job_status":"start","queue_duration_s":0.023698,"scheduling_latency_s":0.023717}2031gitlab # [ 259.821728] gitaly[915]: time="2026-10-04T19:00:34.816Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:43.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.824" pid=915 ref=refs/keep-around/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 remote_ip="2001:db8:1::1" revision=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2032gitlab # [ 259.824849] gitaly[915]: time="2026-10-04T19:00:34.816Z" level=info msg="optimized repository" clean_stale_data_time=149.18125ms clean_work_trees_time="70.12µs" commit_graph_time="4.47µs" correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.469µs" pid=915 prune_time="3.631µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.749µs" storage_name=default system=housekeeping total_time=149.397759ms2033gitlab # [ 259.865868] gitaly[915]: time="2026-10-04T19:00:34.816Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:00:43.113" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:00:33.824" pid=915 ref=refs/keep-around/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 remote_ip="2001:db8:1::1" revision=7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 span.kind=server system=grpc username=root2034gitlab # [ 259.866646] gitaly[915]: time="2026-10-04T19:00:34.836Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M3SZRZHBJ4M6Z415ZAK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:04.834" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:34.835" grpc.time_ms=0.438 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2035gitlab # [ 259.867455] nixbot[2691]: {"ts": "2026-10-04T19:00:34.849Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-7edc24b17daf-5", "argv0": "bwrap"}2036gitlab # [ 259.867936] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.843Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"eadf220546a377a1bd6551cd","created_at":"2026-10-04T19:00:34.732Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-c2a4f27f98df4e3e","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.735Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-eadf220546a377a1bd6551cd: done: 0.084322 sec","job_status":"done","queue_duration_s":0.023698,"scheduling_latency_s":0.023717,"gitaly_calls":2,"gitaly_duration_s":0.00599,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004143,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000615,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000689,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000179,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000142,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000261,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054778,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.064854611,"duration_s":0.084322,"completed_at":"2026-10-04T19:00:34.843Z","load_balancing_strategy":"primary","db_duration_s":0.021795,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2037gitlab # [ 259.923246] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9451392b0112f156a6e2382b","created_at":"2026-10-04T19:00:34.796Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-609586d3c2804ee5","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.797Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9451392b0112f156a6e2382b: start","job_status":"start","queue_duration_s":0.046524,"scheduling_latency_s":0.04654}2038gitlab # [ 259.924027] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:34.865Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9451392b0112f156a6e2382b","created_at":"2026-10-04T19:00:34.796Z","trace_propagation_headers":{"sentry-trace":"eefa980e6a7a4ae19ac5fad60b32849c-609586d3c2804ee5","baggage":"sentry-trace_id=eefa980e6a7a4ae19ac5fad60b32849c,sentry-release=Unknown"},"correlation_id":"01M444M3SZRZHBJ4M6Z415ZAK7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:34.797Z","job_size_bytes":3,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9451392b0112f156a6e2382b: done: 0.021323 sec","job_status":"done","queue_duration_s":0.046524,"scheduling_latency_s":0.04654,"redis_calls":7,"redis_duration_s":0.001015,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021323,"completed_at":"2026-10-04T19:00:34.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab # [ 260.052382] nix-daemon[2863]: accepted connection from pid 3721, user nixbot2040gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses2041gitlab # [ 260.519478] gitaly[915]: time="2026-10-04T19:00:35.527Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M5G8APWT473ZWS0JBZA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:05.225" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:35.526" grpc.time_ms=1.042 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2042gitlab # [ 260.627057] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1702043gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses, in 0.22 seconds)2044gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses2045gitlab # [ 261.745716] gitaly[915]: time="2026-10-04T19:00:36.753Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M6P55T53C8F11WBKY178 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:06.750" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:36.751" grpc.time_ms=1.064 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2046gitlab # [ 261.789167] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1182047gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses, in 0.16 seconds)2048gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses2049gitlab # [ 262.897598] gitaly[915]: time="2026-10-04T19:00:37.904Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M7TG523Q4EM5QJ8ACH0E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:07.002" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:37.903" grpc.time_ms=0.972 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2050gitlab # [ 262.945356] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1122051gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses, in 0.15 seconds)2052gitlab # [ 263.260583] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"7aba7f696f7b698a4adcd364","created_at":"2026-10-04T19:00:38.249Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.264Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-7aba7f696f7b698a4adcd364: start","job_status":"start","queue_duration_s":0.002607,"scheduling_latency_s":0.002651}2053gitlab # [ 263.368496] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"7aba7f696f7b698a4adcd364","created_at":"2026-10-04T19:00:38.249Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.264Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-7aba7f696f7b698a4adcd364: done: 0.108197 sec","job_status":"done","queue_duration_s":0.002607,"scheduling_latency_s":0.002651,"redis_calls":24,"redis_duration_s":0.034036,"redis_read_bytes":24,"redis_write_bytes":5977,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006083,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000581,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3810,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.011874,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.015498,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001093156,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.108197,"completed_at":"2026-10-04T19:00:38.375Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 263.442961] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"e3d15a054f201b4e16164335","created_at":"2026-10-04T19:00:38.277Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3a032069d6af19ed05c260510c3b9a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.292Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-e3d15a054f201b4e16164335: start","job_status":"start","queue_duration_s":0.086004,"scheduling_latency_s":0.086047}2055gitlab # [ 263.482802] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"e3d15a054f201b4e16164335","created_at":"2026-10-04T19:00:38.277Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3a032069d6af19ed05c260510c3b9a6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.292Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-e3d15a054f201b4e16164335: done: 0.111932 sec","job_status":"done","queue_duration_s":0.086004,"scheduling_latency_s":0.086047,"redis_calls":12,"redis_duration_s":0.005595,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000904584,"exclusive_lock_hold_duration_s":0.08318208,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.111932,"completed_at":"2026-10-04T19:00:38.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007728,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}2056gitlab # [ 263.524982] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"651b0255b6f16dd26de5498c","created_at":"2026-10-04T19:00:38.305Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a1cd18ee7cb760dec8cb52f0beca377","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.316Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-651b0255b6f16dd26de5498c: start","job_status":"start","queue_duration_s":0.216291,"scheduling_latency_s":0.216315}2057gitlab # [ 263.568656] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"651b0255b6f16dd26de5498c","created_at":"2026-10-04T19:00:38.305Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a1cd18ee7cb760dec8cb52f0beca377","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.316Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-651b0255b6f16dd26de5498c: done: 0.043337 sec","job_status":"done","queue_duration_s":0.216291,"scheduling_latency_s":0.216315,"redis_calls":10,"redis_duration_s":0.0052899999999999996,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000563,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001503,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001028063,"duration_s":0.043337,"completed_at":"2026-10-04T19:00:38.575Z","load_balancing_strategy":"primary","db_duration_s":0.010434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2058gitlab # [ 263.614794] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f6fef2e53686fe3199172353","created_at":"2026-10-04T19:00:38.329Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f23439b9f0d45f7cbdc346d90dd23525","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.349Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f6fef2e53686fe3199172353: start","job_status":"start","queue_duration_s":0.273021,"scheduling_latency_s":0.273075}2059gitlab # [ 263.686843] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f6fef2e53686fe3199172353","created_at":"2026-10-04T19:00:38.329Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f23439b9f0d45f7cbdc346d90dd23525","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.349Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f6fef2e53686fe3199172353: done: 0.072158 sec","job_status":"done","queue_duration_s":0.273021,"scheduling_latency_s":0.273075,"redis_calls":9,"redis_duration_s":0.012098000000000001,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003069,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.072158,"completed_at":"2026-10-04T19:00:38.694Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060gitlab # [ 263.739505] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"efbbe39cdb345c14e687425f","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-efbbe39cdb345c14e687425f: start","job_status":"start","queue_duration_s":0.394117,"scheduling_latency_s":0.39417}2061gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses2062gitlab # [ 263.918125] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"efbbe39cdb345c14e687425f","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-efbbe39cdb345c14e687425f: done: 0.178259 sec","job_status":"done","queue_duration_s":0.394117,"scheduling_latency_s":0.39417,"redis_calls":18,"redis_duration_s":0.014183,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002957,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.006692,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.102,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.101,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001167187,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.178259,"completed_at":"2026-10-04T19:00:38.924Z","load_balancing_strategy":"primary","db_duration_s":0.102096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 263.994144] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"bd637c295022741f4effbd83","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-bd637c295022741f4effbd83: start","job_status":"start","queue_duration_s":0.577201,"scheduling_latency_s":0.577238}2064gitlab # [ 264.005855] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"bd637c295022741f4effbd83","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-bd637c295022741f4effbd83: done: 0.05332 sec","job_status":"done","queue_duration_s":0.577201,"scheduling_latency_s":0.577238,"redis_calls":15,"redis_duration_s":0.011363999999999999,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003394,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.00797,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":4,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":4,"duration_s":0.05332,"completed_at":"2026-10-04T19:00:38.983Z","load_balancing_strategy":"primary","db_duration_s":0.005447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses, in 0.10 seconds)2066gitlab # [ 264.045164] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:38.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"f2a14368e26d7b163cc48c73","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-f2a14368e26d7b163cc48c73: start","job_status":"start","queue_duration_s":0.634083,"scheduling_latency_s":0.634119}2067gitlab # [ 264.057325] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"f2a14368e26d7b163cc48c73","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-f2a14368e26d7b163cc48c73: done: 0.064214 sec","job_status":"done","queue_duration_s":0.634083,"scheduling_latency_s":0.634119,"redis_calls":15,"redis_duration_s":0.016006,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.010229,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.064214,"completed_at":"2026-10-04T19:00:39.050Z","load_balancing_strategy":"primary","db_duration_s":0.01326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 264.096065] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"d657504a41a996a224b5bd69","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-d657504a41a996a224b5bd69: start","job_status":"start","queue_duration_s":0.702536,"scheduling_latency_s":0.702569}2069gitlab # [ 264.096694] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 622070gitlab # [ 264.097434] gitaly[915]: time="2026-10-04T19:00:39.041Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M8YPW2HF6P1H266XQR2S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:09.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:39.040" grpc.time_ms=0.747 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab # [ 264.100426] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"d657504a41a996a224b5bd69","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-d657504a41a996a224b5bd69: done: 0.052518 sec","job_status":"done","queue_duration_s":0.702536,"scheduling_latency_s":0.702569,"redis_calls":15,"redis_duration_s":0.009214,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.006097,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.052518,"completed_at":"2026-10-04T19:00:39.107Z","load_balancing_strategy":"primary","db_duration_s":0.011167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab # [ 264.157768] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"bb079020f0edc6553a65fde8","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-bb079020f0edc6553a65fde8: start","job_status":"start","queue_duration_s":0.757871,"scheduling_latency_s":0.757903}2073gitlab # [ 264.158558] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"bb079020f0edc6553a65fde8","created_at":"2026-10-04T19:00:38.317Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-a759d2afb94e4702","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"8bccb931536f83f04ac7e61513227b0c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.352Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-bb079020f0edc6553a65fde8: done: 0.036998 sec","job_status":"done","queue_duration_s":0.757871,"scheduling_latency_s":0.757903,"redis_calls":15,"redis_duration_s":0.006725,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.003489,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.036998,"completed_at":"2026-10-04T19:00:39.147Z","load_balancing_strategy":"primary","db_duration_s":0.004337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074gitlab # [ 264.205574] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"a7fae3ec9823e625d223e772","created_at":"2026-10-04T19:00:38.362Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1a56e2b17159a33b380595df53cf993","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.549Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-a7fae3ec9823e625d223e772: start","job_status":"start","queue_duration_s":0.607798,"scheduling_latency_s":0.607834}2075gitlab # [ 264.206281] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"a7fae3ec9823e625d223e772","created_at":"2026-10-04T19:00:38.362Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1a56e2b17159a33b380595df53cf993","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.549Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-a7fae3ec9823e625d223e772: done: 0.039283 sec","job_status":"done","queue_duration_s":0.607798,"scheduling_latency_s":0.607834,"redis_calls":12,"redis_duration_s":0.008166,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005382,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060762,"exclusive_lock_hold_duration_s":0.010535138,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.039283,"completed_at":"2026-10-04T19:00:39.196Z","load_balancing_strategy":"primary","db_duration_s":0.005358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 264.250564] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ccc5fc9f54ef7cb865881e38","created_at":"2026-10-04T19:00:38.556Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87cb85cc87d0f4b5b23ff6b67bd30691","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.567Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ccc5fc9f54ef7cb865881e38: start","job_status":"start","queue_duration_s":0.631264,"scheduling_latency_s":0.631278}2077gitlab # [ 264.251183] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ccc5fc9f54ef7cb865881e38","created_at":"2026-10-04T19:00:38.556Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87cb85cc87d0f4b5b23ff6b67bd30691","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.567Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ccc5fc9f54ef7cb865881e38: done: 0.044574 sec","job_status":"done","queue_duration_s":0.631264,"scheduling_latency_s":0.631278,"redis_calls":12,"redis_duration_s":0.00496,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001751,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000839213,"exclusive_lock_hold_duration_s":0.010426465,"duration_s":0.044574,"completed_at":"2026-10-04T19:00:39.242Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 264.296118] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"16b4ae26d8ecead988071bbc","created_at":"2026-10-04T19:00:38.613Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd8f6b0246a338fc2d52fa14da8e07b5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.628Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-16b4ae26d8ecead988071bbc: start","job_status":"start","queue_duration_s":0.621032,"scheduling_latency_s":0.621055}2079gitlab # [ 264.297055] nix-daemon[2863]: accepted connection from pid 3758, user nixbot2080gitlab # [ 264.297743] nix-daemon[2863]: accepted connection from pid 3721, user nixbot2081gitlab # [ 264.298426] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"16b4ae26d8ecead988071bbc","created_at":"2026-10-04T19:00:38.613Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd8f6b0246a338fc2d52fa14da8e07b5","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.628Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-16b4ae26d8ecead988071bbc: done: 0.042469 sec","job_status":"done","queue_duration_s":0.621032,"scheduling_latency_s":0.621055,"redis_calls":11,"redis_duration_s":0.008021,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005385,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000617396,"exclusive_lock_hold_duration_s":0.007775011,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.042469,"completed_at":"2026-10-04T19:00:39.292Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082gitlab # [ 264.352203] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"eac07c9317af7746f30c5299","created_at":"2026-10-04T19:00:38.641Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19f3f5ee9b0198866c5035a615894f64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:00:38.660Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-eac07c9317af7746f30c5299: start","job_status":"start","queue_duration_s":0.637043,"scheduling_latency_s":0.637059}2083gitlab # [ 264.352890] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.322Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2084gitlab # [ 264.353565] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"eac07c9317af7746f30c5299","created_at":"2026-10-04T19:00:38.641Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"19f3f5ee9b0198866c5035a615894f64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:00:38.660Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-eac07c9317af7746f30c5299: done: 0.047107 sec","job_status":"done","queue_duration_s":0.637043,"scheduling_latency_s":0.637059,"redis_calls":9,"redis_duration_s":0.004168,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00099,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000811,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047107,"completed_at":"2026-10-04T19:00:39.344Z","load_balancing_strategy":"primary","db_duration_s":0.009705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 264.403670] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"e45d5a49665527b41e0cf0e4","created_at":"2026-10-04T19:00:38.670Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11a18fcf39e3dfb5eb8379bca865d54d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.683Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-e45d5a49665527b41e0cf0e4: start","job_status":"start","queue_duration_s":0.665817,"scheduling_latency_s":0.665837}2086gitlab # [ 264.404578] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"e45d5a49665527b41e0cf0e4","created_at":"2026-10-04T19:00:38.670Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11a18fcf39e3dfb5eb8379bca865d54d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.683Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-e45d5a49665527b41e0cf0e4: done: 0.042354 sec","job_status":"done","queue_duration_s":0.665817,"scheduling_latency_s":0.665837,"redis_calls":10,"redis_duration_s":0.007678,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000995,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003032,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001797994,"duration_s":0.042354,"completed_at":"2026-10-04T19:00:39.391Z","load_balancing_strategy":"primary","db_duration_s":0.008679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 264.457691] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"36854e1c7d00f2442c915177","created_at":"2026-10-04T19:00:38.742Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8842e529461b0f5a87e7af87873b05ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:00:38.765Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-36854e1c7d00f2442c915177: start","job_status":"start","queue_duration_s":0.628561,"scheduling_latency_s":0.628579}2088gitlab # [ 264.458842] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"36854e1c7d00f2442c915177","created_at":"2026-10-04T19:00:38.742Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8842e529461b0f5a87e7af87873b05ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:00:38.765Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-36854e1c7d00f2442c915177: done: 0.058511 sec","job_status":"done","queue_duration_s":0.628561,"scheduling_latency_s":0.628579,"redis_calls":9,"redis_duration_s":0.004749,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001018,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058511,"completed_at":"2026-10-04T19:00:39.452Z","load_balancing_strategy":"primary","db_duration_s":0.026627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2089gitlab # [ 264.509499] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.456Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"610440099c75a5a01314f1e1","created_at":"2026-10-04T19:00:38.777Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27cec7a88de669287b2e1cf16762758f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.788Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-610440099c75a5a01314f1e1: start","job_status":"start","queue_duration_s":0.667152,"scheduling_latency_s":0.667171}2090gitlab # [ 264.510057] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.463Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2091gitlab # [ 264.516188] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"610440099c75a5a01314f1e1","created_at":"2026-10-04T19:00:38.777Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27cec7a88de669287b2e1cf16762758f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.788Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-610440099c75a5a01314f1e1: done: 0.067867 sec","job_status":"done","queue_duration_s":0.667152,"scheduling_latency_s":0.667171,"redis_calls":9,"redis_duration_s":0.001212,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067867,"completed_at":"2026-10-04T19:00:39.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092gitlab # [ 264.563856] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"a70cf89fd47445a4ee32549c","created_at":"2026-10-04T19:00:38.888Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b87683659bccbd1b452f1154f2ab9a42","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.903Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-a70cf89fd47445a4ee32549c: start","job_status":"start","queue_duration_s":0.621956,"scheduling_latency_s":0.62198}2093gitlab # [ 264.564696] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.556Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"a70cf89fd47445a4ee32549c","created_at":"2026-10-04T19:00:38.888Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b87683659bccbd1b452f1154f2ab9a42","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.903Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-a70cf89fd47445a4ee32549c: done: 0.031602 sec","job_status":"done","queue_duration_s":0.621956,"scheduling_latency_s":0.62198,"redis_calls":11,"redis_duration_s":0.006737,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001625,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000570743,"duration_s":0.031602,"completed_at":"2026-10-04T19:00:39.556Z","load_balancing_strategy":"primary","db_duration_s":0.001175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2094gitlab # [ 264.621562] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a4fa87342d639e844ab68c3a","created_at":"2026-10-04T19:00:38.926Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52fc9d61c745ee2f66b0c675d138fe47","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.933Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a4fa87342d639e844ab68c3a: start","job_status":"start","queue_duration_s":0.631469,"scheduling_latency_s":0.631488}2095gitlab # [ 264.622225] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.581Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2096gitlab # [ 264.622683] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a4fa87342d639e844ab68c3a","created_at":"2026-10-04T19:00:38.926Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52fc9d61c745ee2f66b0c675d138fe47","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.933Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a4fa87342d639e844ab68c3a: done: 0.028266 sec","job_status":"done","queue_duration_s":0.631469,"scheduling_latency_s":0.631488,"redis_calls":9,"redis_duration_s":0.005262,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004183,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028266,"completed_at":"2026-10-04T19:00:39.593Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2097gitlab # [ 264.668862] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"74da337268e6b6392716d420","created_at":"2026-10-04T19:00:38.948Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f0bcaa1471d796cadba4be25870d21e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.956Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-74da337268e6b6392716d420: start","job_status":"start","queue_duration_s":0.638349,"scheduling_latency_s":0.638364}2098gitlab # [ 264.669708] gitaly[915]: time="2026-10-04T19:00:39.583Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M9E18BX8FWVAMVZB83PV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:09.281" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:39.582" grpc.time_ms=1.38 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2099gitlab # [ 264.670455] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"74da337268e6b6392716d420","created_at":"2026-10-04T19:00:38.948Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f0bcaa1471d796cadba4be25870d21e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.956Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-74da337268e6b6392716d420: done: 0.029045 sec","job_status":"done","queue_duration_s":0.638349,"scheduling_latency_s":0.638364,"redis_calls":18,"redis_duration_s":0.004712,"redis_read_bytes":20,"redis_write_bytes":4694,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000187,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003426,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029045,"completed_at":"2026-10-04T19:00:39.623Z","load_balancing_strategy":"primary","db_duration_s":0.001555,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100gitlab # [ 264.714291] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"f4bf7d8753b3f014ec448685","created_at":"2026-10-04T19:00:38.964Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4d436bf0c12e6e740401f3e16c4bc73","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.996Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-f4bf7d8753b3f014ec448685: start","job_status":"start","queue_duration_s":0.628563,"scheduling_latency_s":0.628581}2101gitlab # [ 264.715851] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"f4bf7d8753b3f014ec448685","created_at":"2026-10-04T19:00:38.964Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4d436bf0c12e6e740401f3e16c4bc73","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:38.996Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-f4bf7d8753b3f014ec448685: done: 0.039497 sec","job_status":"done","queue_duration_s":0.628563,"scheduling_latency_s":0.628581,"redis_calls":11,"redis_duration_s":0.001803,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000382,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000164,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199746,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.039497,"completed_at":"2026-10-04T19:00:39.664Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 264.760604] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"fe7330e5df7b03c8982c15e3","created_at":"2026-10-04T19:00:39.008Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0ded9fd7ef4901335774fefd6b8fc9b","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.019Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-fe7330e5df7b03c8982c15e3: start","job_status":"start","queue_duration_s":0.646288,"scheduling_latency_s":0.646302}2103gitlab # [ 264.801809] gitaly[915]: time="2026-10-04T19:00:39.706Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444M9E18BX8FWVAMVZB83PV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:49.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:39.700" grpc.time_ms=5.703 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab # [ 264.802696] nixbot[2691]: {"ts": "2026-10-04T19:00:39.752Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 \"HTTP/1.1 201 Created\""}2105gitlab # [ 264.804316] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"fe7330e5df7b03c8982c15e3","created_at":"2026-10-04T19:00:39.008Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0ded9fd7ef4901335774fefd6b8fc9b","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.019Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-fe7330e5df7b03c8982c15e3: done: 0.020012 sec","job_status":"done","queue_duration_s":0.646288,"scheduling_latency_s":0.646302,"redis_calls":10,"redis_duration_s":0.001439,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215391,"duration_s":0.020012,"completed_at":"2026-10-04T19:00:39.685Z","load_balancing_strategy":"primary","db_duration_s":0.005559,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 264.845794] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.686Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"a7cd9272b1f80eac7406d717","created_at":"2026-10-04T19:00:39.029Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f82b436dabd792919fb04a67eb8716d2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.048Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-a7cd9272b1f80eac7406d717: start","job_status":"start","queue_duration_s":0.638033,"scheduling_latency_s":0.638047}2107gitlab # [ 264.846495] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2612108gitlab # [ 264.846993] gitaly[915]: time="2026-10-04T19:00:39.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M9PB77N85VDFQFX3YWRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:09.498" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:39.798" grpc.time_ms=0.619 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2109gitlab # [ 264.847826] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"a7cd9272b1f80eac7406d717","created_at":"2026-10-04T19:00:39.029Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f82b436dabd792919fb04a67eb8716d2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.048Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-a7cd9272b1f80eac7406d717: done: 0.01287 sec","job_status":"done","queue_duration_s":0.638033,"scheduling_latency_s":0.638047,"redis_calls":12,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000249,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000313,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000277968,"duration_s":0.01287,"completed_at":"2026-10-04T19:00:39.699Z","load_balancing_strategy":"primary","db_duration_s":0.00076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 264.889187] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2d789d758c3bff30c8fe3ac7","created_at":"2026-10-04T19:00:39.059Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7306cafb8d8d02bd551e93b4b556e6df","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.070Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2d789d758c3bff30c8fe3ac7: start","job_status":"start","queue_duration_s":0.629724,"scheduling_latency_s":0.629738}2111gitlab # [ 264.899700] gitaly[915]: time="2026-10-04T19:00:39.831Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444M9PB77N85VDFQFX3YWRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:49.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:39.825" grpc.time_ms=6.532 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab # [ 264.912290] nixbot[2691]: {"ts": "2026-10-04T19:00:39.866Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 \"HTTP/1.1 201 Created\""}2113gitlab # [ 264.952517] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2d789d758c3bff30c8fe3ac7","created_at":"2026-10-04T19:00:39.059Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7306cafb8d8d02bd551e93b4b556e6df","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.070Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2d789d758c3bff30c8fe3ac7: done: 0.075338 sec","job_status":"done","queue_duration_s":0.629724,"scheduling_latency_s":0.629738,"redis_calls":70,"redis_duration_s":0.011119,"redis_read_bytes":118,"redis_write_bytes":42430,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001435,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000631,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37335,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008593,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075338,"completed_at":"2026-10-04T19:00:39.775Z","load_balancing_strategy":"primary","db_duration_s":0.00257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 264.955093] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"c75c0ed2fbabf78f07ad3e2c","created_at":"2026-10-04T19:00:39.075Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee0ead17950d89c1b6759b8a559ad962","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.083Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-c75c0ed2fbabf78f07ad3e2c: start","job_status":"start","queue_duration_s":0.693308,"scheduling_latency_s":0.693328}2115gitlab # [ 265.003808] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1092116gitlab # [ 265.004610] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1002117gitlab # [ 265.005152] gitaly[915]: time="2026-10-04T19:00:39.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444M9T69YKQAP63FRDS7XSE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:09.618" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:39.918" grpc.time_ms=0.685 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2118gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses2119gitlab # [ 265.005833] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"c75c0ed2fbabf78f07ad3e2c","created_at":"2026-10-04T19:00:39.075Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee0ead17950d89c1b6759b8a559ad962","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.083Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-c75c0ed2fbabf78f07ad3e2c: done: 0.018531 sec","job_status":"done","queue_duration_s":0.693308,"scheduling_latency_s":0.693328,"redis_calls":10,"redis_duration_s":0.003457,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217066,"duration_s":0.018531,"completed_at":"2026-10-04T19:00:39.795Z","load_balancing_strategy":"primary","db_duration_s":0.000626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120gitlab # [ 265.049960] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e83999b14b1bdecc336b2a82","created_at":"2026-10-04T19:00:39.095Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3feef5ac99b99d66b1d63a2772c7050f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.106Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e83999b14b1bdecc336b2a82: start","job_status":"start","queue_duration_s":0.689632,"scheduling_latency_s":0.689644}2121gitlab # [ 265.050765] nixbot[2691]: {"ts": "2026-10-04T19:00:39.979Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4 \"HTTP/1.1 201 Created\""}2122gitlab # [ 265.051261] gitaly[915]: time="2026-10-04T19:00:39.946Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444M9T69YKQAP63FRDS7XSE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:00:49.119" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:00:39.940" grpc.time_ms=6.192 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2123gitlab # [ 265.051913] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e83999b14b1bdecc336b2a82","created_at":"2026-10-04T19:00:39.095Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3feef5ac99b99d66b1d63a2772c7050f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.106Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e83999b14b1bdecc336b2a82: done: 0.040768 sec","job_status":"done","queue_duration_s":0.689632,"scheduling_latency_s":0.689644,"redis_calls":11,"redis_duration_s":0.0020800000000000003,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000313,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000703162,"exclusive_lock_hold_duration_s":0.018405971,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.040768,"completed_at":"2026-10-04T19:00:39.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 265.088735] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"05bcb18134372e72ddc356b4","created_at":"2026-10-04T19:00:39.115Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c49f0d5290dca9a73a514141921f5a62","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.125Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-05bcb18134372e72ddc356b4: start","job_status":"start","queue_duration_s":0.711953,"scheduling_latency_s":0.711969}2125gitlab # [ 265.129642] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"05bcb18134372e72ddc356b4","created_at":"2026-10-04T19:00:39.115Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c49f0d5290dca9a73a514141921f5a62","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.125Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-05bcb18134372e72ddc356b4: done: 0.01837 sec","job_status":"done","queue_duration_s":0.711953,"scheduling_latency_s":0.711969,"redis_calls":10,"redis_duration_s":0.001398,"redis_read_bytes":9,"redis_write_bytes":1055,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000246121,"duration_s":0.01837,"completed_at":"2026-10-04T19:00:39.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126gitlab # [ 265.169976] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"ad3dc57a667270f5a121c3c9","created_at":"2026-10-04T19:00:39.130Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4474cf2b6517121369139054ba57b473","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.135Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-ad3dc57a667270f5a121c3c9: start","job_status":"start","queue_duration_s":0.721471,"scheduling_latency_s":0.721484}2127gitlab # [ 265.170681] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"ad3dc57a667270f5a121c3c9","created_at":"2026-10-04T19:00:39.130Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4474cf2b6517121369139054ba57b473","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.135Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-ad3dc57a667270f5a121c3c9: done: 0.027045 sec","job_status":"done","queue_duration_s":0.721471,"scheduling_latency_s":0.721484,"redis_calls":20,"redis_duration_s":0.0032449999999999996,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000978,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000575,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036541,"duration_s":0.027045,"completed_at":"2026-10-04T19:00:39.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2128gitlab # [ 265.214724] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.885Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"1d605d867a5813b2d3550ab5","created_at":"2026-10-04T19:00:39.140Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c26ef06c99f5c3d0cee277f8decf8fe8","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.146Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-1d605d867a5813b2d3550ab5: start","job_status":"start","queue_duration_s":0.738954,"scheduling_latency_s":0.738969}2129gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses, in 0.19 seconds)2130(finished: subtest: eval and build post commit statuses, in 5.94 seconds)2131(finished: run the VM test script, in 314.51 seconds)2132gitlab # [ 265.215675] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"1d605d867a5813b2d3550ab5","created_at":"2026-10-04T19:00:39.140Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c26ef06c99f5c3d0cee277f8decf8fe8","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.146Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-1d605d867a5813b2d3550ab5: done: 0.013612 sec","job_status":"done","queue_duration_s":0.738954,"scheduling_latency_s":0.738969,"redis_calls":11,"redis_duration_s":0.001634,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000355,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204775,"duration_s":0.013612,"completed_at":"2026-10-04T19:00:39.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 265.263627] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.900Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"af04ce9b22a865f144b906b5","created_at":"2026-10-04T19:00:39.156Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22b03d98c99eae6809fa7efef94090fe","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.184Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-af04ce9b22a865f144b906b5: start","job_status":"start","queue_duration_s":0.716217,"scheduling_latency_s":0.716231}2134gitlab # [ 265.264871] gitaly[915]: time="2026-10-04T19:00:40.220Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444MA1D98PFNSEY4PC73YVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:01:10.217" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:00:40.218" grpc.time_ms=1.848 pid=915 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2135gitlab # [ 265.269824] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:40 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7edc24b17daf4bacdaa734228bc79d3da0d4e6e4/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1412136gitlab # [ 265.271628] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:19:00:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 932137test script finished in 314.61s2138cleanup2139kill QemuMachine (pid 46)2140gitlab # [ 265.273616] sidekiq-cluster[2570]: {"severity":"INFO","time":"2026-10-04T19:00:39.920Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"af04ce9b22a865f144b906b5","created_at":"2026-10-04T19:00:39.156Z","trace_propagation_headers":{"sentry-trace":"8791e84cf13249268838e59b3bd08ddd-1ac735c1a5ce4a21","baggage":"sentry-trace_id=8791e84cf13249268838e59b3bd08ddd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22b03d98c99eae6809fa7efef94090fe","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:00:39.184Z","job_size_bytes":2,"pid":2570,"sidekiq_tid":"m82m","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-af04ce9b22a865f144b906b5: done: 0.020057 sec","job_status":"done","queue_duration_s":0.716217,"scheduling_latency_s":0.716231,"redis_calls":9,"redis_duration_s":0.002019,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001413,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020057,"completed_at":"2026-10-04T19:00:39.920Z","load_balanqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2141gitlab # c2142(finished: cleanup, in 1.87 seconds)2143additionally exposed symbols:2144 gitlab,2145 vlan1,2146 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2147[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-04T18:59:43.069Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2148[]2149[]2150[]2151[]2152[]2153[]2154[]2155[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-04T19:00:33.722032Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2156[]2157[{'id': 5, 'sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:00:34.724Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2158[{'id': 5, 'sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:00:34.724Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2159[{'id': 5, 'sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:00:34.724Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2160[{'id': 5, 'sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:00:34.724Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2161[{'id': 5, 'sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-10-04T19:00:34.724Z', 'started_at': None, 'finished_at': '2026-10-04T19:00:39.723Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '7edc24b17daf4bacdaa734228bc79d3da0d4e6e4', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-04T19:00:39.846Z', 'started_at': None, 'finished_at': '2026-10-04T19:00:39.959Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]