vm-test-run-nixbot-gitlab
x86_64-linux.nixbot-gitlab
· build #125
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 13)11gitlab # Formatting '/build/vm-state-gitlab/tmp.3co2y06fyf', fmt=raw size=107374182412gitlab # mke2fs 1.47.3 (8-Jul-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 12054fd1-130c-4277-9c60-eac103afd5ed16gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618gitlab # 19(finished: start all VMs, in 46.25 seconds)20gitlab: waiting for unit setup-gitlab.service21gitlab # Allocating group tables: 0/8 done22gitlab: waiting for the VM to finish booting23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d8hrizznh012g54c1z57n0q0lc2m50fy-nixos-system-gitlab-test/init regInfo=/nix/store/fc0zhrxy037ly12zz04d414cy9rkxwpv-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 512434117 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000004] tsc: Detected 2400.006 MHz processor67gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000876] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002736] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002747] Using GB pages for direct mapping73gitlab # [ 0.002830] RAMDISK: [mem 0xbe4e0000-0xbffcffff]74gitlab # [ 0.002837] ACPI: Early table checksum verification disabled75gitlab # [ 0.002840] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002843] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002847] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002854] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002856] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002857] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002859] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002861] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002862] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002863] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002864] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002864] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002865] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002865] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003091] No NUMA configuration found90gitlab # [ 0.003092] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003094] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005407] Zone ranges:93gitlab # [ 0.005408] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005409] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005411] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005411] Device empty97gitlab # [ 0.005412] Movable zone start for each node98gitlab # [ 0.005413] Early memory node ranges99gitlab # [ 0.005413] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005414] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005415] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005416] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005438] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005707] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.114967] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.115389] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.115403] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.115433] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.115436] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.115437] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.115438] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.115439] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.115440] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.115443] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.115444] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.115448] TSC deadline timer available117gitlab # [ 0.115452] CPU topo: Max. logical packages: 1118gitlab # [ 0.115453] CPU topo: Max. logical dies: 1119gitlab # [ 0.115454] CPU topo: Max. dies per package: 1120gitlab # [ 0.115457] CPU topo: Max. threads per core: 1121gitlab # [ 0.115457] CPU topo: Num. cores per package: 4122gitlab # [ 0.115458] CPU topo: Num. threads per package: 4123gitlab # [ 0.115458] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.115482] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.115499] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.115502] kvm-guest: setup PV sched yield127gitlab # [ 0.115518] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.115519] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.115520] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.115522] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.115523] Booting paravirtualized kernel on KVM132gitlab # [ 0.115527] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.119960] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.122191] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.122239] kvm-guest: PV spinlocks enabled136gitlab # [ 0.122241] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.122243] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d8hrizznh012g54c1z57n0q0lc2m50fy-nixos-system-gitlab-test/init regInfo=/nix/store/fc0zhrxy037ly12zz04d414cy9rkxwpv-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.122338] Unknown kernel command line parameters "regInfo=/nix/store/fc0zhrxy037ly12zz04d414cy9rkxwpv-closure-info/registration", will be passed to user space.139gitlab # [ 0.122351] random: crng init done140gitlab # [ 0.122352] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.131372] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.136022] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.136099] software IO TLB: area num 4.144gitlab # [ 0.211673] Fallback order for Node 0: 0145gitlab # [ 0.211681] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.211683] Policy zone: Normal147gitlab # [ 0.214216] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.224155] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.237839] allocated 12582912 bytes of page_ext150gitlab # [ 0.247874] ftrace: allocating 48565 entries in 192 pages151gitlab # [ 0.247875] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.248745] Dynamic Preempt: lazy153gitlab # [ 0.248920] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.248921] rcu: RCU event tracing is enabled.155gitlab # [ 0.248922] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.248924] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.248924] Rude variant of Tasks RCU enabled.158gitlab # [ 0.248924] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.248925] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.248925] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.248951] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.248953] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.248954] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.253292] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.253619] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.253738] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)167gitlab # [ 0.257281] Console: colour VGA+ 80x25168gitlab # [ 0.257284] printk: legacy console [tty0] enabled169gitlab # [ 0.287818] printk: legacy console [ttyS0] enabled170gitlab # [ 0.389236] ACPI: Core revision 20250807171gitlab # [ 0.390104] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns172gitlab # [ 0.391659] APIC: Switch to symmetric I/O mode setup173gitlab # [ 0.392657] x2apic enabled174gitlab # [ 0.393439] APIC: Switched APIC routing to: physical x2apic175gitlab # [ 0.394371] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()176gitlab # [ 0.395593] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()177gitlab # [ 0.397081] kvm-guest: setup PV IPIs178gitlab # [ 0.398683] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1179gitlab # [ 0.399689] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983d4fda2, max_idle_ns: 440795302975 ns180gitlab # [ 0.401412] Calibrating delay loop (skipped) preset value.. 4800.01 BogoMIPS (lpj=2400006)181gitlab # [ 0.403494] x86/cpu: User Mode Instruction Prevention (UMIP) activated182gitlab # [ 0.404547] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127183gitlab # [ 0.405409] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0184gitlab # [ 0.406412] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto185gitlab # [ 0.408409] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl186gitlab # [ 0.410409] Transient Scheduler Attacks: Vulnerable: No microcode187gitlab # [ 0.411409] Spectre V2 : Mitigation: Enhanced / Automatic IBRS188gitlab # [ 0.412409] Speculative Return Stack Overflow: Mitigation: Safe RET189gitlab # [ 0.413409] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization190gitlab # [ 0.414413] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier191gitlab # [ 0.416409] active return thunk: srso_alias_return_thunk192gitlab # [ 0.417430] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'193gitlab # [ 0.418408] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'194gitlab # [ 0.419408] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'195gitlab # [ 0.420408] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'196gitlab # [ 0.421408] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'197gitlab # [ 0.422409] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'198gitlab # [ 0.423408] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'199gitlab # [ 0.425409] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256200gitlab # [ 0.426408] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64201gitlab # [ 0.427408] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512202gitlab # [ 0.428408] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024203gitlab # [ 0.429408] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8204gitlab # [ 0.430408] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.205gitlab # [ 0.460340] Freeing SMP alternatives memory: 44K206gitlab # [ 0.461005] pid_max: default: 32768 minimum: 301207gitlab # [ 0.461507] LSM: initializing lsm=capability,landlock,yama,bpf,ima208gitlab # [ 0.462515] landlock: Up and running.209gitlab # [ 0.463876] Yama: becoming mindful.210gitlab # [ 0.464622] LSM support for eBPF active211gitlab # [ 0.465640] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)212gitlab # [ 0.466557] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)213gitlab # [ 0.469564] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)214gitlab # [ 0.471400] Performance Events: Fam17h+ core perfctr, AMD PMU driver.215gitlab # [ 0.472413] ... version: 2216gitlab # [ 0.473164] ... bit width: 48217gitlab # [ 0.473458] ... generic counters: 6218gitlab # [ 0.474197] ... generic bitmap: 000000000000003f219gitlab # [ 0.475111] ... fixed-purpose counters: 0220gitlab # [ 0.475410] ... fixed-purpose bitmap: 0000000000000000221gitlab # [ 0.476342] ... value mask: 0000ffffffffffff222gitlab # [ 0.477151] ... max period: 00007fffffffffff223gitlab # [ 0.477411] ... global_ctrl mask: 000000000000003f224gitlab # [ 0.478460] signal: max sigframe size: 3376225gitlab # [ 0.479271] rcu: Hierarchical SRCU implementation.226gitlab # [ 0.480062] rcu: Max phase no-delay instances is 400.227gitlab # [ 0.480601] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level228gitlab # [ 0.485915] smp: Bringing up secondary CPUs ...229gitlab # [ 0.486812] smpboot: x86: Booting SMP configuration:230gitlab # [ 0.487420] .... node #0, CPUs: #1 #2 #3231gitlab # [ 0.490535] smp: Brought up 1 node, 4 CPUs232gitlab # [ 0.492114] smpboot: Total of 4 processors activated (19200.04 BogoMIPS)233gitlab # [ 0.492901] Memory: 6013136K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved)234gitlab # [ 0.494411] devtmpfs: initialized235gitlab # [ 0.495432] x86/mm: Memory block size: 128MB236gitlab # [ 0.498552] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns237gitlab # [ 0.499456] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)238gitlab # [ 0.500489] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).239gitlab # [ 0.501541] pinctrl core: initialized pinctrl subsystem240gitlab # [ 0.502680] PM: RTC time: 06:30:47, date: 2026-07-18241gitlab # [ 0.506465] NET: Registered PF_NETLINK/PF_ROUTE protocol family242gitlab # [ 0.508769] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations243gitlab # [ 0.509466] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations244gitlab # [ 0.511518] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations245gitlab # [ 0.512423] audit: initializing netlink subsys (disabled)246gitlab # [ 0.513436] audit: type=2000 audit(1784356247.130:1): state=initialized audit_enabled=0 res=1247gitlab # [ 0.513646] thermal_sys: Registered thermal governor 'fair_share'248gitlab # [ 0.515411] thermal_sys: Registered thermal governor 'bang_bang'249gitlab # [ 0.516411] thermal_sys: Registered thermal governor 'step_wise'250gitlab # [ 0.517411] thermal_sys: Registered thermal governor 'user_space'251gitlab # [ 0.518404] thermal_sys: Registered thermal governor 'power_allocator'252gitlab # [ 0.519424] cpuidle: using governor menu253gitlab # [ 0.521619] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5254gitlab # [ 0.522651] PCI: Using configuration type 1 for base access255gitlab # [ 0.523412] PCI: Using configuration type 1 for extended access256gitlab # [ 0.524611] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.257gitlab # [ 0.527721] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages258gitlab # [ 0.528412] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page259gitlab # [ 0.529412] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages260gitlab # [ 0.530412] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page261gitlab # [ 0.535839] ACPI: Added _OSI(Module Device)262gitlab # [ 0.537422] ACPI: Added _OSI(Processor Device)263gitlab # [ 0.539417] ACPI: Added _OSI(Processor Aggregator Device)264gitlab # [ 0.546238] ACPI: 1 ACPI AML tables successfully acquired and loaded265gitlab # [ 0.553447] ACPI: Interpreter enabled266gitlab # [ 0.554473] ACPI: PM: (supports S0 S3 S4 S5)267gitlab # [ 0.556418] ACPI: Using IOAPIC for interrupt routing268gitlab # [ 0.558556] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug269gitlab # [ 0.562419] PCI: Using E820 reservations for host bridge windows270gitlab # [ 0.565775] ACPI: Enabled 2 GPEs in block 00 to 0F271gitlab # [ 0.578671] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])272gitlab # [ 0.580426] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]273gitlab # [ 0.584156] acpiphp: Slot [3] registered274gitlab # [ 0.585442] acpiphp: Slot [4] registered275gitlab # [ 0.587464] acpiphp: Slot [5] registered276gitlab # [ 0.588479] acpiphp: Slot [6] registered277gitlab # [ 0.589464] acpiphp: Slot [7] registered278gitlab # [ 0.591464] acpiphp: Slot [8] registered279gitlab # [ 0.592478] acpiphp: Slot [9] registered280gitlab # [ 0.594464] acpiphp: Slot [10] registered281gitlab # [ 0.595462] acpiphp: Slot [11] registered282gitlab # [ 0.596464] acpiphp: Slot [12] registered283gitlab # [ 0.598480] acpiphp: Slot [13] registered284gitlab # [ 0.599455] acpiphp: Slot [14] registered285gitlab # [ 0.600387] acpiphp: Slot [15] registered286gitlab # [ 0.600954] acpiphp: Slot [16] registered287gitlab # [ 0.602439] acpiphp: Slot [17] registered288gitlab # [ 0.603146] acpiphp: Slot [18] registered289gitlab # [ 0.603430] acpiphp: Slot [19] registered290gitlab # [ 0.604158] acpiphp: Slot [20] registered291gitlab # [ 0.604440] acpiphp: Slot [21] registered292gitlab # [ 0.605166] acpiphp: Slot [22] registered293gitlab # [ 0.606431] acpiphp: Slot [23] registered294gitlab # [ 0.607145] acpiphp: Slot [24] registered295gitlab # [ 0.607431] acpiphp: Slot [25] registered296gitlab # [ 0.608157] acpiphp: Slot [26] registered297gitlab # [ 0.609431] acpiphp: Slot [27] registered298gitlab # [ 0.610152] acpiphp: Slot [28] registered299gitlab # [ 0.610437] acpiphp: Slot [29] registered300gitlab # [ 0.611190] acpiphp: Slot [30] registered301gitlab # [ 0.612431] acpiphp: Slot [31] registered302gitlab # [ 0.613128] PCI host bridge to bus 0000:00303gitlab # [ 0.613422] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]304gitlab # [ 0.614446] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]305gitlab # [ 0.616415] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]306gitlab # [ 0.617412] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]307gitlab # [ 0.618413] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]308gitlab # [ 0.621444] pci_bus 0000:00: root bus resource [bus 00-ff]309gitlab # [ 0.624242] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint310gitlab # [ 0.628792] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint311gitlab # [ 0.633544] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint312gitlab # [ 0.640502] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]313gitlab # [ 0.642485] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk314gitlab # [ 0.645417] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk315gitlab # [ 0.648416] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk316gitlab # [ 0.650415] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk317gitlab # [ 0.652794] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint318gitlab # [ 0.658473] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]319gitlab # [ 0.662549] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint320gitlab # [ 0.665959] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI321gitlab # [ 0.667418] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB322gitlab # [ 0.668642] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint323gitlab # [ 0.683458] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]324gitlab # [ 0.684423] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]325gitlab # [ 0.685460] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]326gitlab # [ 0.687625] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]327gitlab # [ 0.689385] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint328gitlab # [ 0.694445] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]329gitlab # [ 0.695346] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]330gitlab # [ 0.696200] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]331gitlab # [ 0.709378] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]332gitlab # [ 0.711644] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint333gitlab # [ 0.715389] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]334gitlab # [ 0.716104] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]335gitlab # [ 0.717434] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]336gitlab # [ 0.719653] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint337gitlab # [ 0.725047] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]338gitlab # [ 0.726418] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]339gitlab # [ 0.727433] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]340gitlab # [ 0.729710] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint341gitlab # [ 0.733442] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]342gitlab # [ 0.734336] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]343gitlab # [ 0.735197] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]344gitlab # [ 0.737630] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint345gitlab # [ 0.741444] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]346gitlab # [ 0.742342] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]347gitlab # [ 0.743205] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]348gitlab # [ 0.745652] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint349gitlab # [ 0.749412] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]350gitlab # [ 0.750315] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]351gitlab # [ 0.751198] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]352gitlab # [ 0.754616] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint353gitlab # [ 0.763439] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]354gitlab # [ 0.765434] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]355gitlab # [ 0.767473] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]356gitlab # [ 0.771433] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]357gitlab # [ 0.777634] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint358gitlab # [ 0.785623] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]359gitlab # [ 0.788472] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]360gitlab # [ 0.794606] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint361gitlab # [ 0.803455] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]362gitlab # [ 0.805432] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]363gitlab # [ 0.808495] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]364gitlab # [ 0.814260] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint365gitlab # [ 0.822439] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]366gitlab # [ 0.823405] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]367gitlab # [ 0.824188] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]368gitlab # [ 0.829461] ACPI: PCI: Interrupt link LNKA configured for IRQ 10369gitlab # [ 0.830568] ACPI: PCI: Interrupt link LNKB configured for IRQ 10370gitlab # [ 0.831548] ACPI: PCI: Interrupt link LNKC configured for IRQ 11371gitlab # [ 0.832541] ACPI: PCI: Interrupt link LNKD configured for IRQ 11372gitlab # [ 0.833487] ACPI: PCI: Interrupt link LNKS configured for IRQ 9373gitlab # [ 0.836720] iommu: Default domain type: Translated374gitlab # [ 0.837413] iommu: DMA domain TLB invalidation policy: lazy mode375gitlab # [ 0.838694] ACPI: bus type USB registered376gitlab # [ 0.839488] usbcore: registered new interface driver usbfs377gitlab # [ 0.840430] usbcore: registered new interface driver hub378gitlab # [ 0.841368] usbcore: registered new device driver usb379gitlab # [ 0.844274] NetLabel: Initializing380gitlab # [ 0.844849] NetLabel: domain hash size = 128381gitlab # [ 0.845411] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO382gitlab # [ 0.846403] NetLabel: unlabeled traffic allowed by default383gitlab # [ 0.847142] PCI: Using ACPI for IRQ routing384gitlab # [ 0.848641] pci 0000:00:02.0: vgaarb: setting as boot VGA device385gitlab # [ 0.849407] pci 0000:00:02.0: vgaarb: bridge control possible386gitlab # [ 0.849407] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none387gitlab # [ 0.851415] vgaarb: loaded388gitlab # [ 0.852141] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0389gitlab # [ 0.853411] hpet0: 3 comparators, 64-bit 100.000000 MHz counter390gitlab # [ 0.858496] clocksource: Switched to clocksource kvm-clock391gitlab # [ 0.860162] VFS: Disk quotas dquot_6.6.0392gitlab # [ 0.860925] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)393gitlab # [ 0.862662] pnp: PnP ACPI init394gitlab # [ 0.863802] pnp: PnP ACPI: found 6 devices395gitlab # [ 0.873487] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns396gitlab # [ 0.875071] clocksource: Switched to clocksource acpi_pm397gitlab # [ 0.876089] NET: Registered PF_INET protocol family398gitlab # [ 0.878256] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)399gitlab # [ 0.896772] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)400gitlab # [ 0.898313] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)401gitlab # [ 0.899710] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)402gitlab # [ 0.903430] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)403gitlab # [ 0.904801] TCP: Hash tables configured (established 65536 bind 65536)404gitlab # [ 0.905978] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)405gitlab # [ 0.907316] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)406gitlab # [ 0.908753] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.910123] NET: Registered PF_UNIX/PF_LOCAL protocol family408gitlab # [ 0.911115] NET: Registered PF_XDP protocol family409gitlab # [ 0.911989] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]410gitlab # [ 0.913027] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]411gitlab # [ 0.914058] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]412gitlab # [ 0.915200] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]413gitlab # [ 0.916370] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]414gitlab # [ 0.917722] pci 0000:00:01.0: PIIX3: Enabling Passive Release415gitlab # [ 0.918722] pci 0000:00:00.0: Limiting direct PCI/PCI transfers416gitlab # [ 0.921117] ACPI: \_SB_.LNKD: Enabled at IRQ 11417gitlab # [ 0.923644] PCI: CLS 0 bytes, default 64418gitlab # [ 0.924413] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)419gitlab # [ 0.925034] Trying to unpack rootfs image as initramfs...420gitlab # [ 0.925492] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)421gitlab # [ 0.928248] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983d4fda2, max_idle_ns: 440795302975 ns422gitlab # [ 0.948555] Initialise system trusted keyrings423gitlab # [ 0.949731] workingset: timestamp_bits=40 max_order=21 bucket_order=0424gitlab # [ 0.961157] Key type asymmetric registered425gitlab # [ 0.961938] Asymmetric key parser 'x509' registered426gitlab # [ 0.962854] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)427gitlab # [ 0.964468] io scheduler mq-deadline registered428gitlab # [ 0.965295] io scheduler kyber registered429gitlab # [ 0.969310] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled430gitlab # [ 0.970635] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A431gitlab # [ 0.973067] Linux agpgart interface v0.103432gitlab # [ 0.973873] ACPI: bus type drm_connector registered433gitlab # [ 0.976422] usbcore: registered new interface driver usbserial_generic434gitlab # [ 0.977539] usbserial: USB Serial support registered for generic435gitlab # [ 0.978563] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.436gitlab # [ 0.978563] Please enable it if your BIOS has the CPPC option.437gitlab # [ 0.980858] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled438gitlab # [ 0.982271] drop_monitor: Initializing network drop monitor service439gitlab # [ 0.983458] NET: Registered PF_INET6 protocol family440gitlab # [ 0.985362] Segment Routing with IPv6441gitlab # [ 0.986121] In-situ OAM (IOAM) with IPv6442gitlab # [ 0.988140] IPI shorthand broadcast: enabled443gitlab # [ 0.992558] sched_clock: Marking stable (852015911, 139782110)->(1011704120, -19906099)444gitlab # [ 0.994380] registered taskstats version 1445gitlab # [ 0.995464] Loading compiled-in X.509 certificates446gitlab # [ 1.005645] Demotion targets for Node 0: null447gitlab # [ 1.006694] Key type .fscrypt registered448gitlab # [ 1.007397] Key type fscrypt-provisioning registered449gitlab # [ 1.008385] ima: No TPM chip found, activating TPM-bypass!450gitlab # [ 1.009334] ima: Allocated hash algorithm: sha1451gitlab # [ 1.010224] ima: No architecture policies found452gitlab # [ 1.011283] PM: Magic number: 2:471:519453gitlab # [ 1.012903] RAS: Correctable Errors collector initialized.454gitlab # [ 1.017395] clk: Disabling unused clocks455gitlab # [ 1.018148] PM: genpd: Disabling unused power domains456gitlab # [ 1.197570] Freeing initrd memory: 27584K457gitlab # [ 1.201167] Freeing unused decrypted memory: 2028K458gitlab # [ 1.204005] Freeing unused kernel image (initmem) memory: 3644K459gitlab # [ 1.205121] Write protecting the kernel read-only data: 32768k460gitlab # [ 1.207314] Freeing unused kernel image (text/rodata gap) memory: 1332K461gitlab # [ 1.209018] Freeing unused kernel image (rodata/data gap) memory: 816K462gitlab # [ 1.259438] x86/mm: Checked W+X mappings: passed, no W+X pages found.463gitlab # [ 1.260598] Run /init as init process464gitlab # [ 1.272077] systemd[1]: Inserted module 'autofs4'465gitlab # [ 1.293377] fuse: init (API version 7.45)466gitlab # [ 1.301579] ACPI: \_SB_.LNKC: Enabled at IRQ 10467gitlab # [ 1.308142] ACPI: \_SB_.LNKA: Enabled at IRQ 10468gitlab # [ 1.311248] ACPI: \_SB_.LNKB: Enabled at IRQ 11469gitlab # [ 1.346707] systemd[1]: Successfully made /usr/ read-only.470gitlab # [ 1.684063] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)471gitlab # [ 1.696411] systemd[1]: Detected virtualization kvm.472gitlab # [ 1.727401] systemd[1]: Detected architecture x86-64.473gitlab # [ 1.729625] systemd[1]: Running in initrd.474gitlab # [ 1.732244] systemd[1]: Initializing machine ID from random generator.475gitlab # [ 1.735154] systemd[1]: Hostname set to <gitlab>.476gitlab # [ 1.801344] systemd[1]: Queued start job for default target Initrd Default Target.477gitlab # [ 1.932697] systemd[1]: Created slice Slice /system/modprobe.478gitlab # [ 1.933963] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.479gitlab # [ 1.935394] systemd[1]: Expecting device /dev/disk/by-label/nix-store...480gitlab # [ 1.937702] systemd[1]: Expecting device /dev/disk/by-label/nixos...481gitlab # [ 1.940494] systemd[1]: Reached target Path Units.482gitlab # [ 1.942664] systemd[1]: Reached target Slice Units.483gitlab # [ 1.944929] systemd[1]: Reached target Swaps.484gitlab # [ 1.946950] systemd[1]: Reached target Timer Units.485gitlab # [ 1.949362] systemd[1]: Listening on D-Bus System Message Bus Socket.486gitlab # [ 1.952342] systemd[1]: Listening on Journal Socket (/dev/log).487gitlab # [ 1.955154] systemd[1]: Listening on Journal Sockets.488gitlab # [ 1.957443] systemd[1]: Listening on udev Control Socket.489gitlab # [ 1.959469] systemd[1]: Listening on udev Kernel Socket.490gitlab # [ 1.961396] systemd[1]: Reached target Socket Units.491gitlab # [ 1.967452] systemd[1]: Starting Create List of Static Device Nodes...492gitlab # [ 1.973682] systemd[1]: Starting Load Kernel Module 9pnet_virtio...493gitlab # [ 1.983368] systemd[1]: Starting Load Kernel Module configfs...494gitlab # [ 1.990727] systemd[1]: Starting Journal Service...495gitlab # [ 1.995416] systemd[1]: Starting Load Kernel Modules...496gitlab # [ 1.997851] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki497gitlab # [ 2.001332] systemd[1]: Starting Coldplug All udev Devices...498gitlab # [ 2.031589] netfs: FS-Cache loaded499gitlab # [ 2.033430] systemd[1]: Finished Create List of Static Device Nodes.500gitlab # [ 2.036504] systemd[1]: modprobe@configfs.service: Deactivated successfully.501gitlab # [ 2.042374] systemd[1]: Finished Load Kernel Module configfs.502gitlab # [ 2.045140] 9pnet: Installing 9P2000 support503gitlab # [ 2.046021] systemd-journald[152]: Collecting audit messages is disabled.504gitlab # [ 2.049320] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.505gitlab # [ 2.051683] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev506gitlab # [ 2.054081] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config507gitlab # [ 2.057797] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...508gitlab # [ 2.059469] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.509gitlab # [ 2.069310] systemd[1]: Finished Load Kernel Module 9pnet_virtio.510gitlab # [ 2.074036] systemd[1]: Finished Load Kernel Modules.511gitlab # [ 2.076655] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.512gitlab # [ 2.080054] systemd[1]: Starting Apply Kernel Variables...513gitlab # [ 2.082761] systemd[1]: Starting Create Static Device Nodes in /dev...514gitlab # [ 2.089287] systemd[1]: Started Journal Service.515gitlab # [ 1.949562] systemd-modules-load[154]: Using 4 probe threads516gitlab # [ 1.951633] systemd-modules-load[154]: Inserted module 'virtio_balloon'517gitlab # [ 1.952934] systemd-modules-load[154]: Inserted module 'virtio_gpu'518gitlab # [ 1.954074] systemd-modules-load[154]: Inserted module 'dm_mod'519gitlab # [ 1.955123] systemd[1]: Finished Create Static Device Nodes in /dev.520gitlab # [ 1.956866] systemd[1]: Finished Apply Kernel Variables.521gitlab # [ 1.960873] systemd[1]: Reached target Preparation for Local File Systems.522gitlab # [ 1.961870] systemd[1]: Reached target Local File Systems.523gitlab # [ 1.964212] systemd[1]: Starting Create System Files and Directories...524gitlab # [ 1.966359] systemd[1]: Starting Rule-based Manager for Device Events and Files...525gitlab # [ 1.979855] systemd[1]: Finished Create System Files and Directories.526gitlab # [ 1.987459] systemd-udevd[192]: Using default interface naming scheme 'v260'.527gitlab # [ 1.999669] systemd[1]: Started Rule-based Manager for Device Events and Files.528gitlab # [ 2.030150] systemd[1]: Finished Coldplug All udev Devices.529gitlab # [ 2.031040] systemd[1]: Reached target System Initialization.530gitlab # [ 2.031834] systemd[1]: Reached target Basic System.531gitlab # [ 2.245771] virtio_blk virtio4: 4/0/0 default/read/poll queues532gitlab # [ 2.253247] SCSI subsystem initialized533gitlab # [ 2.262877] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)534gitlab # [ 2.283295] virtio_blk virtio5: 4/0/0 default/read/poll queues535gitlab # [ 2.285169] uhci_hcd 0000:00:01.2: UHCI Host Controller536gitlab # [ 2.286991] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1537gitlab # [ 2.288728] uhci_hcd 0000:00:01.2: detected 2 ports538gitlab # [ 2.289701] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140539gitlab # [ 2.291148] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18540gitlab # [ 2.292642] virtio_blk virtio5: [vdb] 11450848 512-byte logical blocks (5.86 GB/5.46 GiB)541gitlab # [ 2.292766] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1542gitlab # [ 2.295887] usb usb1: Product: UHCI Host Controller543gitlab # [ 2.296672] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd544gitlab # [ 2.297504] usb usb1: SerialNumber: 0000:00:01.2545gitlab # [ 2.303465] scsi host0: ata_piix546gitlab # [ 2.305501] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12547gitlab # [ 2.305942] hub 1-0:1.0: USB hub found548gitlab # [ 2.306851] scsi host1: ata_piix549gitlab # [ 2.308141] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0550gitlab # [ 2.308246] hub 1-0:1.0: 2 ports detected551gitlab # [ 2.308477] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0552gitlab # [ 2.315200] serio: i8042 KBD port at 0x60,0x64 irq 1553gitlab # [ 2.316021] serio: i8042 AUX port at 0x60,0x64 irq 12554gitlab # [ 2.194027] (udev-worker)[208]: Network interface NamePolicy= disabled on kernel command line.555gitlab # [ 2.198804] (udev-worker)[199]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.556gitlab # [ 2.201321] (udev-worker)[199]: Network interface NamePolicy= disabled on kernel command line.557gitlab # [ 2.204075] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.215705] systemd[1]: Found device /dev/disk/by-label/nix-store.559gitlab # [ 2.220778] systemd[1]: Found device /dev/disk/by-label/nixos.560gitlab # [ 2.221688] systemd[1]: Reached target Initrd Root Device.561gitlab # [ 2.224573] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...562gitlab # [ 2.228838] systemd-vconsole-setup[232]: Configuration of first virtual console was skipped, ignoring remaining ones.563gitlab # [ 2.235543] systemd[1]: Finished Virtual Console Setup.564gitlab # [ 2.250526] systemd-fsck[239]: nixos: clean, 12/65536 files, 13019/262144 blocks565gitlab # [ 2.255417] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.566gitlab # [ 2.465288] ata2: found unknown device (class 0)567gitlab # [ 2.467151] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100568gitlab # [ 2.471039] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5569gitlab # [ 2.501746] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0570gitlab # [ 2.530874] usb 1-1: new full-speed USB device number 2 using uhci_hcd571gitlab # [ 2.539892] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray572gitlab # [ 2.565431] cdrom: Uniform CD-ROM driver Revision: 3.20573gitlab # [ 2.699517] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00574gitlab # [ 2.702551] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10575gitlab # [ 2.705238] usb 1-1: Product: QEMU USB Tablet576gitlab # [ 2.706891] usb 1-1: Manufacturer: QEMU577gitlab # [ 2.708353] usb 1-1: SerialNumber: 28754-0000:00:01.2-1578gitlab # [ 2.745264] hid: raw HID events driver (C) Jiri Kosina579gitlab # [ 2.756047] usbcore: registered new interface driver usbhid580gitlab # [ 2.758008] usbhid: USB HID core driver581gitlab # [ 2.766481] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2582gitlab # [ 2.769836] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0583gitlab # [ 2.888354] systemd[1]: Mounting /sysroot...584gitlab # [ 3.178940] EXT4-fs (vda): mounted filesystem 12054fd1-130c-4277-9c60-eac103afd5ed r/w with ordered data mode. Quota mode: none.585gitlab # [ 3.044282] systemd[1]: Mounted /sysroot.586gitlab # [ 3.046508] systemd[1]: Reached target Initrd Root File System.587gitlab # [ 3.054610] systemd[1]: Mounting /sysroot/nix/.ro-store...588gitlab # [ 3.059465] systemd[1]: Mounting /sysroot/nix/.rw-store...589gitlab # [ 3.065412] systemd[1]: Mounting /sysroot/run...590gitlab # [ 3.072595] systemd[1]: Mounting /sysroot/tmp/shared...591gitlab # [ 3.219036] erofs (device vdb): mounted with root inode @ nid 128.592gitlab # [ 3.080277] systemd[1]: Mounting /sysroot/tmp/xchg...593gitlab # [ 3.083606] systemd[1]: Starting Mountpoints Configured in the Real Root...594gitlab # [ 3.086312] systemd[1]: Mounted /sysroot/nix/.ro-store.595gitlab # [ 3.087814] systemd[1]: Mounted /sysroot/nix/.rw-store.596gitlab # [ 3.089411] systemd[1]: Mounted /sysroot/run.597gitlab # [ 3.095108] systemd-sysroot-fstab-check[269]: /sysroot should be mounted in the initrd, will request daemon-reload.598gitlab # [ 3.104129] s[ 3.243992] 9p: Installing v9fs 9p2000 file system support599gitlab # ystemd[1]: Starting rw-sysroot-nix-store.service...600gitlab # [ 3.106523] systemd[1]: Reload requested from client PID 269 ('systemd-sysroot') (unit initrd-parse-etc.service)...601gitlab # [ 3.109504] systemd[1]: Reloading...602gitlab # [ 3.354993] systemd[1]: Reloading finished in 245 ms.603gitlab # [ 3.400723] systemd-sysroot-fstab-check[269]: Requesting initrd-fs.target/start/replace...604gitlab # [ 3.492458] systemd[1]: Mounted /sysroot/tmp/shared.605gitlab # [ 3.494383] systemd[1]: Mounted /sysroot/tmp/xchg.606gitlab # [ 3.496167] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.607gitlab # [ 3.498591] systemd[1]: Finished rw-sysroot-nix-store.service.608gitlab # [ 3.504137] systemd-sysroot-fstab-check[269]: Requesting swap.target/start/replace...609gitlab # [ 3.507137] systemd[1]: Starting rw-sysroot-nix-store.service...610gitlab # [ 3.510618] systemd[1]: initrd-parse-etc.service: Deactivated successfully.611gitlab # [ 3.514567] systemd[1]: Finished Mountpoints Configured in the Real Root.612gitlab # [ 3.517069] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.613gitlab # [ 3.529195] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.614gitlab # [ 3.532472] systemd[1]: Finished rw-sysroot-nix-store.service.615gitlab # [ 3.888569] systemd[1]: Mounting /sysroot/nix/store...616gitlab # [ 3.937130] systemd[1]: Mounted /sysroot/nix/store.617gitlab # [ 3.939270] systemd[1]: Reached target Initrd File Systems.618gitlab # [ 3.944049] systemd[1]: Starting Find NixOS closure...619gitlab # [ 3.948435] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...620gitlab # [ 3.977605] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.621gitlab # [ 3.980566] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.622gitlab # [ 3.994296] systemd[1]: Finished Find NixOS closure.623gitlab # [ 3.996186] systemd[1]: Reached target Initrd Default Target.624gitlab # [ 4.000628] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...625gitlab # [ 4.030617] systemd[1]: Stopped target Initrd Default Target.626gitlab # [ 4.032682] systemd[1]: Stopped target Basic System.627gitlab # [ 4.034440] systemd[1]: Stopped target Initrd Root Device.628gitlab # [ 4.036300] systemd[1]: Stopped target Path Units.629gitlab # [ 4.038104] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.630gitlab # [ 4.040615] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.631gitlab # [ 4.043167] systemd[1]: Stopped target Slice Units.632gitlab # [ 4.045318] systemd[1]: Stopped target Socket Units.633gitlab # [ 4.047125] systemd[1]: Stopped target System Initialization.634gitlab # [ 4.049110] systemd[1]: Stopped target Swaps.635gitlab # [ 4.050658] systemd[1]: Stopped target Timer Units.636gitlab # [ 4.052362] systemd[1]: dbus.socket: Deactivated successfully.637gitlab # [ 4.054337] systemd[1]: Closed D-Bus System Message Bus Socket.638gitlab # [ 4.056361] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.639gitlab # [ 4.058799] systemd[1]: Stopped Find NixOS closure.640gitlab # [ 4.060455] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio641gitlab # [ 4.063976] systemd[1]: Starting rw-sysroot-nix-store.service...642gitlab # [ 4.065995] systemd[1]: systemd-sysctl.service: Deactivated successfully.643gitlab # [ 4.068410] systemd[1]: Stopped Apply Kernel Variables.644gitlab # [ 4.070558] systemd[1]: systemd-modules-load.service: Deactivated successfully.645gitlab # [ 4.073329] systemd[1]: Stopped Load Kernel Modules.646gitlab # [ 4.074937] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.647gitlab # [ 4.077129] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.648gitlab # [ 4.079134] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.649gitlab # [ 4.081046] systemd[1]: Stopped Create System Files and Directories.650gitlab # [ 4.084928] systemd[1]: Stopped target Local File Systems.651gitlab # [ 4.086407] systemd[1]: Stopped target Preparation for Local File Systems.652gitlab # [ 4.088179] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.653gitlab # [ 4.090053] systemd[1]: Stopped Coldplug All udev Devices.654gitlab # [ 4.093599] systemd[1]: Stopping Rule-based Manager for Device Events and Files...655gitlab # [ 4.096057] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.656gitlab # [ 4.097632] systemd[1]: Stopped Virtual Console Setup.657gitlab # [ 4.106580] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.658gitlab # [ 4.109338] systemd[1]: Finished rw-sysroot-nix-store.service.659gitlab # [ 4.114421] systemd[1]: systemd-udevd.service: Deactivated successfully.660gitlab # [ 4.117248] systemd[1]: Stopped Rule-based Manager for Device Events and Files.661gitlab # [ 4.126942] systemd[1]: initrd-cleanup.service: Deactivated successfully.662gitlab # [ 4.129886] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.663gitlab # [ 4.139267] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.664gitlab # [ 4.140533] systemd[1]: Closed udev Control Socket.665gitlab # [ 4.145809] systemd[1]: Starting Cleanup udev Database...666gitlab # [ 4.146819] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.667gitlab # [ 4.148132] systemd[1]: Stopped Create Static Device Nodes in /dev.668gitlab # [ 4.151595] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.669gitlab # [ 4.152948] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.670gitlab # [ 4.157544] systemd[1]: kmod-static-nodes.service: Deactivated successfully.671gitlab # [ 4.158726] systemd[1]: Stopped Create List of Static Device Nodes.672gitlab # [ 4.165055] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.673gitlab # [ 4.168528] systemd[1]: Finished Cleanup udev Database.674gitlab # [ 4.172849] systemd[1]: Reached target Switch Root.675gitlab # [ 4.175345] systemd[1]: Starting NixOS Activation...676gitlab # [ 4.303870] initrd-nixos-activation-start[571]: booting system configuration /nix/store/d8hrizznh012g54c1z57n0q0lc2m50fy-nixos-system-gitlab-test677gitlab # [ 4.344273] initrd-nixos-activation-start[571]: running activation script...678gitlab # [ 4.594125] initrd-nixos-activation-start[594]: setting up /etc...679gitlab # [ 4.676146] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.680gitlab # [ 4.680509] systemd[1]: Finished NixOS Activation.681gitlab # [ 4.690993] systemd[1]: Starting Switch Root...682gitlab # [ 4.724491] systemd[1]: Switching root.683gitlab # [ 5.179975] systemd-journald[152]: Received SIGTERM from PID 1 (systemd).684gitlab # [ 5.309133] NET: Registered PF_VSOCK protocol family685gitlab # [ 5.677020] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)686gitlab # [ 5.687342] systemd[1]: Detected virtualization kvm.687gitlab # [ 5.689257] systemd[1]: Detected architecture x86-64.688gitlab # [ 5.691225] systemd[1]: Detected first boot.689gitlab # [ 5.693912] systemd[1]: Initializing machine ID from random generator.690gitlab # [ 5.836374] systemd[1]: bpf-restrict-fs: LSM BPF program attached691gitlab # [ 5.897667] systemd[1]: Applying preset policy.692gitlab # [ 6.038709] systemd[1]: Populated /etc with preset unit settings.693gitlab # [ 6.142880] systemd[1]: initrd-switch-root.service: Deactivated successfully.694gitlab # [ 6.144263] systemd[1]: Stopped initrd-switch-root.service.695gitlab # [ 6.146297] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.696gitlab # [ 6.148398] systemd[1]: Created slice Slice /system/getty.697gitlab # [ 6.149870] systemd[1]: Created slice GitLab DevOps Platform Slice.698gitlab # [ 6.151609] systemd[1]: Created slice User and Session Slice.699gitlab # [ 6.152562] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.700gitlab # [ 6.153897] systemd[1]: Started Forward Password Requests to Wall Directory Watch.701gitlab # [ 6.155105] systemd[1]: Expecting device /dev/hvc0...702gitlab # [ 6.155876] systemd[1]: Expecting device /dev/ttyS0...703gitlab # [ 6.156628] systemd[1]: Reached target Local Encrypted Volumes.704gitlab # [ 6.157500] systemd[1]: Stopped target initrd-fs.target.705gitlab # [ 6.158345] systemd[1]: Stopped target initrd-root-fs.target.706gitlab # [ 6.159246] systemd[1]: Stopped target initrd-switch-root.target.707gitlab # [ 6.160216] systemd[1]: Reached target Virtual Machines and Containers.708gitlab # [ 6.161184] systemd[1]: Reached target Path Units.709gitlab # [ 6.161944] systemd[1]: Reached target Remote File Systems.710gitlab # [ 6.162834] systemd[1]: Reached target Slice Units.711gitlab # [ 6.163570] systemd[1]: Reached target Swaps.712gitlab # [ 6.165205] systemd[1]: Listening on Process Core Dump Socket.713gitlab # [ 6.166588] systemd[1]: Listening on Credential Encryption/Decryption.714gitlab # [ 6.168958] systemd[1]: Starting Journal Log Access Socket...715gitlab # [ 6.170383] systemd[1]: Listening on Journal Audit Socket.716gitlab # [ 6.171454] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.717gitlab # [ 6.172631] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki718gitlab # [ 6.174288] systemd[1]: Listening on udev Control Socket.719gitlab # [ 6.178992] systemd[1]: Mounting Huge Pages File System...720gitlab # [ 6.181091] systemd[1]: Mounting POSIX Message Queue File System...721gitlab # [ 6.183358] systemd[1]: Mounting Kernel Debug File System...722gitlab # [ 6.185249] systemd[1]: Mounting Kernel Trace File System...723gitlab # [ 6.195228] systemd[1]: Starting Create List of Static Device Nodes...724gitlab # [ 6.196409] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio725gitlab # [ 6.199347] systemd[1]: Starting Load Kernel Module configfs...726gitlab # [ 6.200352] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm727gitlab # [ 6.201945] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore728gitlab # [ 6.203694] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse729gitlab # [ 6.206426] systemd[1]: Mounting FUSE Control File System...730gitlab # [ 6.208880] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67731gitlab # [ 6.217874] systemd[1]: Starting Journal Service...732gitlab # [ 6.220647] systemd[1]: Starting Load Kernel Modules...733gitlab # [ 6.225121] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...734gitlab # [ 6.233094] systemd[1]: Starting Remount Root and Kernel File Systems...735gitlab # [ 6.235625] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki736gitlab # [ 6.241624] systemd[1]: Starting Coldplug All udev Devices...737gitlab # [ 6.251385] systemd-journald[788]: Collecting audit messages is enabled.738gitlab # [ 6.270360] systemd[1]: Listening on Journal Log Access Socket.739gitlab # [ 6.271411] systemd[1]: Mounted Huge Pages File System.740gitlab # [ 6.272399] systemd[1]: Mounted POSIX Message Queue File System.741gitlab # [ 6.273512] systemd[1]: Mounted Kernel Debug File System.742gitlab # [ 6.274647] systemd[1]: Mounted Kernel Trace File System.743gitlab # [ 6.276121] systemd[1]: Finished Create List of Static Device Nodes.744gitlab # [ 6.277623] systemd[1]: modprobe@configfs.service: Deactivated successfully.745gitlab # [ 6.279249] systemd[1]: Finished Load Kernel Module configfs.746gitlab # [ 6.279424] EXT4-fs (vda): re-mounted 12054fd1-130c-4277-9c60-eac103afd5ed.747gitlab # [ 6.279735] systemd[1]: Mounted FUSE Control File System.748gitlab # [ 6.141329] systemd[1]: Queued start job for default target Multi-User System.749gitlab # [ 6.282585] loop: module loaded750gitlab # [ 6.282965] systemd[1]: Started Journal Service.751gitlab # [ 6.144812] systemd[1]: Unnecessary job was removed for /dev/vdb.752gitlab # [ 6.146141] systemd[1]: systemd-journald.service: Deactivated successfully.753gitlab # [ 6.147344] systemd-modules-load[789]: Using 3 probe threads754gitlab # [ 6.148628] systemd-modules-load[789]: Inserted module 'tls'755gitlab # [ 6.149698] systemd-modules-load[789]: Inserted module 'loop'756gitlab # [ 6.150686] systemd[1]: Finished Load Kernel Modules.757gitlab # [ 6.151590] systemd[1]: Finished Remount Root and Kernel File Systems.758gitlab # [ 6.152636] systemd[1]: Mounting Kernel Configuration File System...759gitlab # [ 6.154333] systemd[1]: Starting Firewall...760gitlab # [ 6.156376] systemd[1]: Starting Flush Journal to Persistent Storage...761gitlab # [ 6.157419] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore762gitlab # [ 6.165238] systemd[1]: Starting Load/Save OS Random Seed...763gitlab # [ 6.168364] systemd-oomd[790]: No swap; memory pressure usage will be degraded764gitlab # [ 6.309687] systemd-journald[788]: Received client request to flush runtime journal.765gitlab # [ 6.213457] systemd[1]: Starting Apply Kernel Variables...766gitlab # [ 6.214988] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...767gitlab # [ 6.216421] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki768gitlab # [ 6.217789] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.769gitlab # [ 6.220152] systemd[1]: Mounted Kernel Configuration File System.770gitlab # [ 6.221062] systemd[1]: Finished Apply Kernel Variables.771gitlab # [ 6.221782] systemd[1]: Finished Load/Save OS Random Seed.772gitlab # [ 6.222783] systemd[1]: Reached target First Boot Complete.773gitlab # [ 6.223707] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.774gitlab # [ 6.224768] systemd[1]: Starting Create Static Device Nodes in /dev...775gitlab # [ 6.225794] systemd[1]: Finished Flush Journal to Persistent Storage.776gitlab # [ 6.226785] systemd[1]: Finished Create Static Device Nodes in /dev.777gitlab # [ 6.227765] systemd[1]: Reached target Preparation for Local File Systems.778gitlab # [ 6.228755] systemd[1]: Starting Rule-based Manager for Device Events and Files...779gitlab # [ 6.264387] systemd-udevd[833]: Using default interface naming scheme 'v260'.780gitlab # [ 6.296628] systemd[1]: Started Rule-based Manager for Device Events and Files.781gitlab # [ 6.308063] systemd[1]: Finished Coldplug All udev Devices.782gitlab # [ 6.371138] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse783gitlab # [ 6.400636] (udev-worker)[899]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.784gitlab # [ 6.402830] (udev-worker)[899]: Network interface NamePolicy= disabled on kernel command line.785gitlab # [ 6.404106] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.786gitlab # [ 6.407538] (udev-worker)[905]: Network interface NamePolicy= disabled on kernel command line.787gitlab # [ 6.411840] systemd[1]: Finished Firewall.788gitlab # [ 6.415054] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.789gitlab # [ 6.416240] systemd[1]: Condition check resulted in Virtio network device being skipped.790gitlab # [ 6.418447] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore791gitlab # [ 6.420240] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67792gitlab # [ 6.422547] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore793gitlab # [ 6.424505] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki794gitlab # [ 6.425893] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki795gitlab # [ 6.610716] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3796gitlab # [ 6.616611] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0797gitlab # [ 6.616969] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console798gitlab # [ 6.630955] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD799gitlab # [ 6.633441] Console: switching to colour dummy device 80x25800gitlab # [ 6.635945] ACPI: button: Power Button [PWRF]801gitlab # [ 6.636445] [drm] Found bochs VGA, ID 0xb0c5.802gitlab # [ 6.637386] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.803gitlab # [ 6.641444] mousedev: PS/2 mouse device common for all mice804gitlab # [ 6.644175] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic805gitlab # [ 6.645083] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0806gitlab # [ 6.658398] fbcon: bochs-drmdrmfb (fb0) is primary device807gitlab # [ 6.659582] rtc_cmos 00:05: RTC can wake from S4808gitlab # [ 6.661621] parport_pc 00:03: reported by Plug and Play ACPI809gitlab # [ 6.661707] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]810gitlab # [ 6.661949] rtc_cmos 00:05: registered as rtc0811gitlab # [ 6.662032] rtc_cmos 00:05: setting system clock to 2026-07-18T06:30:53 UTC (1784356253)812gitlab # [ 6.662143] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs813gitlab # [ 6.670836] Floppy drive(s): fd0 is 2.88M AMI BIOS814gitlab # [ 6.672478] Console: switching to colour frame buffer device 160x50815gitlab # [ 6.673084] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5816gitlab # [ 6.674237] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4817gitlab # [ 6.675814] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device818gitlab # [ 6.676272] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6819gitlab # [ 6.685616] FDC 0 is a S82078B820gitlab # [ 6.587299] systemd[1]: Starting Virtual Console Setup...821gitlab # [ 6.617058] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.822gitlab # [ 6.618135] systemd[1]: Stopped Virtual Console Setup.823gitlab # [ 6.639538] systemd[1]: Starting Virtual Console Setup...824gitlab # [ 6.652977] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.825gitlab # [ 6.654220] systemd[1]: Stopped Virtual Console Setup.826gitlab # [ 6.655028] systemd[1]: Starting Virtual Console Setup...827gitlab # [ 6.802809] ppdev: user-space parallel port driver828gitlab # [ 6.823595] kvm_amd: TSC scaling supported829gitlab # [ 6.824381] kvm_amd: Nested Virtualization enabled830gitlab # [ 6.825266] kvm_amd: Nested Paging enabled831gitlab # [ 6.825946] kvm_amd: LBR virtualization supported832gitlab # [ 6.826637] kvm_amd: Virtual VMLOAD VMSAVE supported833gitlab # [ 6.827388] kvm_amd: Virtual GIF supported834gitlab # [ 6.828032] kvm_amd: Virtual NMI enabled835gitlab # [ 6.714396] systemd-vconsole-setup[964]: Configuration of first virtual console was skipped, ignoring remaining ones.836gitlab # [ 6.718538] systemd[1]: Finished Virtual Console Setup.837gitlab # [ 6.861704] EDAC MC: Ver: 3.0.0838gitlab # [ 7.008445] systemd[1]: Mounting /run/wrappers...839gitlab # [ 7.035007] systemd[1]: Mounted /run/wrappers.840gitlab # [ 7.036836] systemd[1]: Reached target Local File Systems.841gitlab # [ 7.038855] systemd[1]: Listening on Boot Loader Control Service Socket.842gitlab # [ 7.041297] systemd[1]: Starting register-nix-paths.service...843gitlab # [ 7.043477] systemd[1]: Starting Create SUID/SGID Wrappers...844gitlab # [ 7.045578] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.845gitlab # [ 7.048669] systemd[1]: Starting Save Transient machine-id to Disk...846gitlab # [ 7.054357] systemd[1]: Starting Create System Files and Directories...847gitlab # [ 7.081922] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.848gitlab # [ 7.083306] systemd[1]: Finished Save Transient machine-id to Disk.849gitlab # [ 7.090429] systemd[1]: Finished Create System Files and Directories.850gitlab # [ 7.094499] systemd[1]: Starting Rebuild Journal Catalog...851gitlab # [ 7.095651] systemd[1]: Starting Record System Boot/Shutdown in UTMP...852gitlab # [ 7.121063] systemd[1]: Finished Record System Boot/Shutdown in UTMP.853gitlab # [ 7.129537] systemd[1]: Finished Rebuild Journal Catalog.854gitlab # [ 7.131960] systemd[1]: Starting Update is Completed...855gitlab # [ 7.144546] systemd[1]: Finished Update is Completed.856gitlab # [ 7.173376] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.857gitlab # [ 7.174582] systemd[1]: Finished Create SUID/SGID Wrappers.858gitlab # [ 7.175539] systemd[1]: suid-sgid-wrappers.service: Consumed 110ms CPU time over 130ms wall clock time, 4.8M memory peak, 1.1M read from disk.859gitlab # [ 7.428706] systemd[1]: Finished register-nix-paths.service.860gitlab # [ 7.431100] systemd[1]: Reached target System Initialization.861gitlab # [ 7.433294] systemd[1]: Started Discard unused filesystem blocks once a week.862gitlab # [ 7.435831] systemd[1]: Started logrotate.timer.863gitlab # [ 7.437685] systemd[1]: Started Daily Cleanup of Temporary Directories.864gitlab # [ 7.440145] systemd[1]: Reached target Timer Units.865gitlab # [ 7.442116] systemd[1]: Listening on D-Bus System Message Bus Socket.866gitlab # [ 7.444457] systemd[1]: Listening on Nix Daemon Socket.867gitlab # [ 7.446467] systemd[1]: Starting nixbot CI service socket...868gitlab # [ 7.448568] systemd[1]: Listening on Hostname Service Socket.869gitlab # [ 7.450708] systemd[1]: Listening on nixbot CI service socket.870gitlab # [ 7.452897] systemd[1]: Reached target Socket Units.871gitlab # [ 7.454912] systemd[1]: Starting D-Bus System Message Bus...872gitlab # [ 7.514980] dbus-broker-launch[1054]: Looking up NSS user entry for 'systemd-timesync'...873gitlab # [ 7.518341] dbus-broker-launch[1054]: NSS returned no entry for 'systemd-timesync'874gitlab # [ 7.521286] dbus-broker-launch[1054]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"875gitlab # [ 7.526560] systemd[1]: Started D-Bus System Message Bus.876gitlab # [ 7.545922] dbus-broker-launch[1054]: Ready877gitlab # [ 7.548443] systemd[1]: Reached target Basic System.878gitlab # [ 7.551367] systemd[1]: Started backdoor.service.879gitlab # [ 7.556140] systemd[1]: Starting gitlab-config.service...880gitlab # [ 7.558270] systemd[1]: Starting Import lastlog data into lastlog2 database...881gitlab # [ 7.569325] systemd[1]: Starting Logrotate configuration check...882gitlab # [ 7.579663] systemd[1]: Starting Name Service Cache Daemon (nsncd)...883gitlab # [ 7.588354] systemd[1]: Starting Post-Boot Actions...884gitlab # connecting to host...885gitlab # [ 7.593584] systemd[1]: Started Reset console on configuration changes.886gitlab: Guest shell says: b'Spawning backdoor root shell...\n'887gitlab: connected to guest root shell888gitlab # [ 7.600500] logrotate[1059]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.889gitlab: (connecting took 8.25 seconds)890gitlab # [ 7.604361] logrotate[1059]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf891gitlab: (finished: waiting for the VM to finish booting, in 8.25 seconds)892gitlab # [ 7.606787] logrotate[1059]: note: 'monthly' overrides previously specified 'weekly'893gitlab # [ 7.608751] systemd[1]: Starting resolvconf update...894gitlab # [ 7.610500] logrotate[1059]: Reading state from file: /var/lib/logrotate.status895gitlab # [ 7.612506] logrotate[1059]: state file /var/lib/logrotate.status does not exist896gitlab # [ 7.616150] logrotate[1059]: Allocating hash table for state file, size 64 entries897gitlab # [ 7.618192] logrotate[1059]: Handling 4 logs898gitlab # [ 7.618809] logrotate[1059]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed899gitlab # [ 7.622398] logrotate[1059]: considering log /var/log/btmp900gitlab # [ 7.623682] logrotate[1059]: Creating new state901gitlab # [ 7.624735] logrotate[1059]: Now: 2026-07-18 06:30902gitlab # [ 7.625536] logrotate[1059]: Last rotated at 2026-07-18 06:00903gitlab # [ 7.626504] logrotate[1059]: log does not need rotating (log has already been rotated)904gitlab # [ 7.627726] logrotate[1059]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed905gitlab # [ 7.629784] logrotate[1059]: considering log /var/log/wtmp906gitlab # [ 7.630602] logrotate[1059]: Creating new state907gitlab # [ 7.631573] logrotate[1059]: Now: 2026-07-18 06:30908gitlab # [ 7.632410] logrotate[1059]: Last rotated at 2026-07-18 06:00909gitlab # [ 7.632791] logrotate[1059]: log does not need rotating (log has already been rotated)910gitlab # [ 7.633456] logrotate[1059]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed911gitlab # [ 7.633954] logrotate[1059]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1059)912gitlab # [ 7.634634] logrotate[1059]: considering log /var/gitlab/state/log/*.log913gitlab # [ 7.635147] logrotate[1059]: log /var/gitlab/state/log/*.log does not exist -- skipping914gitlab # [ 7.635645] logrotate[1059]: Creating new state915gitlab # [ 7.636165] logrotate[1059]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1059)916gitlab # [ 7.636660] logrotate[1059]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed917gitlab # [ 7.637155] logrotate[1059]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1059)918gitlab # [ 7.637627] logrotate[1059]: considering log /var/log/nginx/*.log919gitlab # [ 7.638164] logrotate[1059]: log /var/log/nginx/*.log does not exist -- skipping920gitlab # [ 7.638677] logrotate[1059]: Creating new state921gitlab # [ 7.639402] logrotate[1059]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1059)922gitlab # [ 7.640195] systemd[1]: Finished Logrotate configuration check.923gitlab # [ 7.640901] nsncd[1060]: Jul 18 06:30:54.608 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"924gitlab # [ 7.641815] systemd[1]: Finished Post-Boot Actions.925gitlab # [ 7.642627] systemd[1]: Finished Import lastlog data into lastlog2 database.926gitlab # [ 7.643365] systemd[1]: Started Name Service Cache Daemon (nsncd).927gitlab # [ 7.643937] systemd[1]: Reached target Host and Network Name Lookups.928gitlab # [ 7.644602] systemd[1]: Reached target User and Group Name Lookups.929gitlab # [ 7.653228] systemd[1]: Starting User Login Management...930gitlab # [ 7.667561] systemd[1]: Stopped target Host and Network Name Lookups.931gitlab # [ 7.668576] systemd[1]: Stopping Host and Network Name Lookups...932gitlab # [ 7.669538] systemd[1]: Stopped target User and Group Name Lookups.933gitlab # [ 7.670492] systemd[1]: Stopping User and Group Name Lookups...934gitlab # [ 7.672543] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...935gitlab # [ 7.673561] systemd[1]: nscd.service: Deactivated successfully.936gitlab # [ 7.676683] systemd[1]: Stopped Name Service Cache Daemon (nsncd).937gitlab # [ 7.677681] systemd[1]: nscd.service: Consumed 18ms CPU time over 92ms wall clock time, 8M memory peak, 5.6M read from disk.938gitlab # [ 7.682315] systemd[1]: Starting Name Service Cache Daemon (nsncd)...939gitlab # [ 7.698624] nsncd[1142]: Jul 18 06:30:54.676 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"940gitlab # [ 7.700879] systemd[1]: Started Name Service Cache Daemon (nsncd).941gitlab # [ 7.701863] systemd[1]: Reached target Host and Network Name Lookups.942gitlab # [ 7.702892] systemd[1]: Reached target User and Group Name Lookups.943gitlab # [ 7.704997] systemd-logind[1096]: New seat seat0.944gitlab # [ 7.710416] systemd-logind[1096]: Watching system buttons on /dev/input/event2 (Power Button)945gitlab # [ 7.711704] systemd-logind[1096]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)946gitlab # [ 7.713057] systemd-logind[1096]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)947gitlab # [ 7.714400] systemd[1]: Started User Login Management.948gitlab # [ 7.715230] systemd[1]: Starting linger-users.service...949gitlab # [ 7.721475] systemd[1]: Finished resolvconf update.950gitlab # [ 7.722591] systemd[1]: Reached target Preparation for Network.951gitlab # [ 7.727258] systemd[1]: Starting DHCP Client...952gitlab # [ 7.729261] systemd[1]: Starting Address configuration of eth1...953gitlab # [ 7.731414] systemd[1]: Starting Extra networking commands....954gitlab # [ 7.732417] systemd[1]: linger-users.service: Deactivated successfully.955gitlab # [ 7.738134] systemd[1]: Finished linger-users.service.956gitlab # [ 7.754247] network-addresses-eth1-start[1171]: adding address 192.168.1.1/24... done957gitlab # [ 7.757935] network-addresses-eth1-start[1171]: adding address 2001:db8:1::1/64... done958gitlab # [ 7.762612] systemd[1]: Finished Address configuration of eth1.959gitlab # [ 7.788932] systemd[1]: Finished Extra networking commands..960gitlab # [ 7.790513] systemd[1]: Reached target Network.961gitlab # [ 7.792442] systemd[1]: Starting gitlab-workhorse.service...962gitlab # [ 7.794461] systemd[1]: Starting Nginx Web Server...963gitlab # [ 7.796703] systemd[1]: Starting PostgreSQL Server...964gitlab # [ 7.799875] systemd[1]: Starting Redis Server - redis-gitlab...965gitlab # [ 7.802090] systemd[1]: Starting Permit User Sessions...966gitlab # [ 7.808567] dhcpcd[1189]: dhcpcd-10.3.2 starting967gitlab # [ 7.812728] dhcpcd[1235]: dev: loaded udev968gitlab # [ 7.957033] 8021q: 802.1Q VLAN Support v1.8969gitlab # [ 7.957711] 8021q: adding VLAN 0 to HW filter on device eth1970gitlab # [ 7.818312] systemd[1]: Finished Permit User Sessions.971gitlab # [ 7.823552] systemd[1]: Started Getty on tty1.972gitlab # [ 7.824788] systemd[1]: Reached target Login Prompts.973gitlab # [ 7.830413] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.974gitlab # [ 8.040920] cfg80211: Loading compiled-in X.509 certificates for regulatory database975gitlab # [ 8.046551] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'976gitlab # [ 8.047845] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'977gitlab # [ 8.049989] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2978gitlab # [ 8.051371] cfg80211: failed to load regulatory.db979gitlab # [ 7.922881] nginx-pre-start[1256]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok980gitlab # [ 7.925031] nginx-pre-start[1256]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful981gitlab # [ 7.926919] redis-server[1254]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.982gitlab # [ 7.929127] redis-server[1254]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo983gitlab # [ 7.930542] redis-server[1254]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1254, just started984gitlab # [ 7.932489] systemd[1]: Started Redis Server - redis-gitlab.985gitlab # [ 7.933621] redis-server[1254]: Configuration loaded986gitlab # [ 7.935031] redis[1254]: monotonic clock: POSIX clock_gettime987gitlab # [ 7.936056] redis[1254]: Running mode=standalone, port=0.988gitlab # [ 7.937124] redis[1254]: Server initialized989gitlab # [ 7.937867] systemd[1]: Started Nginx Web Server.990gitlab # [ 7.938664] redis[1254]: Ready to accept connections unix991gitlab # [ 7.939647] redis[1254]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.992gitlab # [ 7.942777] postgresql-pre-start[1257]: The files belonging to this database system will be owned by user "postgres".993gitlab # [ 7.943208] postgresql-pre-start[1257]: This user must also own the server process.994gitlab # [ 7.948630] postgresql-pre-start[1257]: The database cluster will be initialized with locale "en_US.UTF-8".995gitlab # [ 7.950073] postgresql-pre-start[1257]: The default database encoding has acc[ 8.089897] 8021q: adding VLAN 0 to HW filter on device eth0996gitlab # ordingly been set to "UTF8".997gitlab # [ 7.952264] postgresql-pre-start[1257]: The default text search configuration will be set to "english".998gitlab # [ 7.953687] postgresql-pre-start[1257]: Data page checksums are disabled.999gitlab # [ 7.954781] postgresql-pre-start[1257]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1000gitlab # [ 7.956265] postgresql-pre-start[1257]: creating subdirectories ... ok1001gitlab # [ 7.957319] postgresql-pre-start[1257]: selecting dynamic shared memory implementation ... posix1002gitlab # [ 7.957791] dhcpcd[1235]: eth0: waiting for carrier1003gitlab # [ 7.958558] dhcpcd[1235]: eth0: carrier acquired1004gitlab # [ 7.958959] dhcpcd[1235]: DUID 00:01:00:01:31:ed:da:1e:52:54:00:12:34:561005gitlab # [ 7.959627] dhcpcd[1235]: eth0: IAID 00:12:34:561006gitlab # [ 7.960158] dhcpcd[1235]: eth0: adding address fe80::5054:ff:fe12:34561007gitlab # [ 7.971507] postgresql-pre-start[1257]: selecting default "max_connections" ... 1001008gitlab # [ 8.002347] postgresql-pre-start[1257]: selecting default "shared_buffers" ... 128MB1009gitlab # [ 8.230030] systemd[1]: Started gitlab-workhorse.service.1010gitlab # [ 8.361580] postgresql-pre-start[1257]: selecting default time zone ... UTC1011gitlab # [ 8.364555] postgresql-pre-start[1257]: creating configuration files ... ok1012gitlab # [ 8.471204] gitlab-workhorse[1290]: time="2026-07-18T06:30:55Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41013gitlab # [ 8.471724] gitlab-workhorse[1290]: time="2026-07-18T06:30:55Z" level=info msg="Using redis/go-redis"1014gitlab # [ 8.473802] gitlab-workhorse[1290]: time="2026-07-18T06:30:55Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1015gitlab # [ 8.523552] postgresql-pre-start[1257]: running bootstrap script ... ok1016gitlab # [ 8.957268] postgresql-pre-start[1257]: performing post-bootstrap initialization ... ok1017gitlab # [ 9.262298] postgresql-pre-start[1257]: syncing data to disk ... ok1018gitlab # [ 9.262989] postgresql-pre-start[1257]: initdb: warning: enabling "trust" authentication for local connections1019gitlab # [ 9.263771] postgresql-pre-start[1257]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1020gitlab # [ 9.264465] postgresql-pre-start[1257]: Success. You can now start the database server using:1021gitlab # [ 9.265168] postgresql-pre-start[1257]: pg_ctl -D /var/lib/postgresql/17 -l logfile start1022gitlab # [ 9.379579] postgres[1315]: [1315] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1023gitlab # [ 9.380410] postgres[1315]: [1315] LOG: listening on IPv6 address "::1", port 54321024gitlab # [ 9.380969] postgres[1315]: [1315] LOG: listening on IPv4 address "127.0.0.1", port 54321025gitlab # [ 9.383555] postgres[1315]: [1315] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1026gitlab # [ 9.384805] dhcpcd[1235]: eth0: soliciting a DHCP lease1027gitlab # [ 9.393867] postgres[1326]: [1326] LOG: database system was shut down at 202[ 9.533913] NET: Registered PF_PACKET protocol family1028gitlab # 6-07-18 06:30:55 GMT1029gitlab # [ 9.403220] postgres[1315]: [1315] LOG: database system is ready to accept connections1030gitlab # [ 9.405791] dhcpcd[1235]: eth0: offered 10.0.2.15 from 10.0.2.21031gitlab # [ 9.407307] dhcpcd[1235]: eth0: probing address 10.0.2.15/241032gitlab # [ 9.409743] systemd[1]: Started PostgreSQL Server.1033gitlab # [ 9.413524] systemd[1]: Starting PostgreSQL Setup Scripts...1034gitlab # [ 9.511307] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1322]: mkdir -p /var/gitlab/state/home/.ssh: OK1035gitlab # [ 9.514136] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1322]: chmod 700 /var/gitlab/state/home/.ssh: OK1036gitlab # [ 9.554762] postgresql-setup-start[1339]: CREATE DATABASE1037gitlab # [ 9.560653] systemd[1]: Finished gitlab-config.service.1038gitlab # [ 9.565274] systemd[1]: Started gitaly.service.1039gitlab # [ 9.580933] postgresql-setup-start[1357]: CREATE ROLE1040gitlab # [ 9.592441] postgresql-setup-start[1360]: ALTER DATABASE1041gitlab # [ 9.611682] postgresql-setup-start[1365]: CREATE ROLE1042gitlab # [ 9.613640] systemd[1]: Finished PostgreSQL Setup Scripts.1043gitlab # [ 9.614386] systemd[1]: Reached target PostgreSQL.1044gitlab # [ 9.615674] systemd[1]: Starting gitlab-db-config.service...1045gitlab # [ 9.617871] systemd[1]: Starting gitlab-postgresql.service...1046gitlab # [ 9.718609] gitlab-postgresql-start[1375]: CREATE DATABASE1047gitlab # [ 9.757136] gitlab-postgresql-start[1384]: CREATE EXTENSION1048gitlab # [ 9.790651] gitlab-postgresql-start[1395]: CREATE EXTENSION1049gitlab # [ 9.793558] systemd[1]: Finished gitlab-postgresql.service.1050gitlab # [ 10.064637] gitaly[1355]: time="2026-07-18T06:30:57.041Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13551051gitlab # [ 10.064855] gitaly[1355]: time="2026-07-18T06:30:57.042Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13551052gitlab # [ 10.065507] gitaly[1355]: time="2026-07-18T06:30:57.042Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13551053gitlab # [ 10.065960] gitaly[1355]: time="2026-07-18T06:30:57.042Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13551054gitlab # [ 10.066630] gitaly[1355]: time="2026-07-18T06:30:57.042Z" level=info msg="Starting Gitaly" pid=1355 version=18.11.41055gitlab # [ 10.226907] gitaly[1355]: time="2026-07-18T06:30:57.204Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=161 pid=13551056gitlab # [ 10.227527] gitaly[1355]: time="2026-07-18T06:30:57.204Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13551057gitlab # [ 10.228472] gitaly[1355]: time="2026-07-18T06:30:57.204Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13551058gitlab # [ 10.229102] gitaly[1355]: time="2026-07-18T06:30:57.204Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1355/git-exec-4061487238.d/git pid=13551059gitlab # [ 10.233971] gitaly[1355]: time="2026-07-18T06:30:57.211Z" level=info msg="finished detecting git version" duration_ms=6 pid=13551060gitlab # [ 10.234583] gitaly[1355]: time="2026-07-18T06:30:57.211Z" level=info msg="using Git version" pid=1355 version=2.53.g1061gitlab # [ 10.235127] gitaly[1355]: time="2026-07-18T06:30:57.212Z" level=info msg="clearing disk cache object folder" pid=1355 storage=default1062gitlab # [ 10.235548] gitaly[1355]: time="2026-07-18T06:30:57.212Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1592372613 pid=1355 storage=default1063gitlab # [ 10.236060] gitaly[1355]: time="2026-07-18T06:30:57.212Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1592372613 pid=1355 storage=default1064gitlab # [ 10.236655] gitaly[1355]: time="2026-07-18T06:30:57.212Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13551065gitlab # [ 10.237263] gitaly[1355]: time="2026-07-18T06:30:57.212Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13551066gitlab # [ 10.238196] gitaly[1355]: time="2026-07-18T06:30:57.212Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1592372613 clear_duration_ms=0 pid=1355 storage=default1067gitlab # [ 10.284590] gitaly[1355]: time="2026-07-18T06:30:57.262Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13551068gitlab # [ 10.285308] gitaly[1355]: time="2026-07-18T06:30:57.262Z" level=info msg="counting repositories" pid=13551069gitlab # [ 10.285893] gitaly[1355]: time="2026-07-18T06:30:57.263Z" level=info msg="starting to count repositories" pid=1355 storage_path=/var/gitlab/state/repositories1070gitlab # [ 10.286642] gitaly[1355]: time="2026-07-18T06:30:57.263Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1355 storage_path=/var/gitlab/state/repositories1071gitlab # [ 10.287200] gitaly[1355]: time="2026-07-18T06:30:57.263Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13551072gitlab # [ 10.287742] gitaly[1355]: time="2026-07-18T06:30:57.262Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1355 schema=unix1073gitlab # [ 10.288378] gitaly[1355]: time="2026-07-18T06:30:57.262Z" level=info msg="finished tempdir cleaner walk" pid=1355 storage=default time_ms=01074gitlab # [ 10.288840] gitaly[1355]: time="2026-07-18T06:30:57.263Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1355/sock.d/intern pid=1355 schema=unix1075gitlab # [ 10.299768] gitaly[1355]: time="2026-07-18T06:30:57.277Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13551076gitlab # [ 10.300526] gitaly[1355]: time="2026-07-18T06:30:57.277Z" level=info msg="Started Gitaly" duration_ms=235 pid=13551077gitlab # [ 10.301083] gitaly[1355]: time="2026-07-18T06:30:57.278Z" level=info msg="maintenance: daily scheduled" pid=1355 scheduled="2026-07-18 12:00:00 +0000 UTC"1078gitlab # [ 10.633232] dhcpcd[1235]: eth0: soliciting an IPv6 router1079gitlab # [ 10.633954] dhcpcd[1235]: eth0: Router Advertisement from fe80::21080gitlab # [ 10.635198] dhcpcd[1235]: eth0: adding address fec0::5054:ff:fe12:3456/641081gitlab # [ 10.636406] dhcpcd[1235]: eth0: adding route to fec0::/641082gitlab # [ 10.637442] dhcpcd[1235]: eth0: adding default route via fe80::21083gitlab # [ 11.065164] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1370]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1084gitlab # [ 11.065759] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1370]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1085gitlab # [ 11.105151] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1370]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1086gitlab # [ 12.238411] gitaly[1355]: time="2026-07-18T06:30:59.215Z" level=info msg="License database preloaded" duration_ms=2172 pid=13551087gitlab # [ 14.537789] dhcpcd[1235]: eth0: leased 10.0.2.15 for 86400 seconds1088gitlab # [ 14.538251] dhcpcd[1235]: eth0: adding route to 10.0.2.0/241089gitlab # [ 14.539347] dhcpcd[1235]: eth0: adding default route via 10.0.2.21090gitlab # [ 14.628073] systemd[1]: Started DHCP Client.1091gitlab # [ 14.630809] systemd[1]: Reached target Network is Online.1092gitlab # [ 39.875690] postgres[1732]: [1732] ERROR: relation "feature_gates" does not exist at character 5231093gitlab # [ 39.876039] postgres[1732]: [1732] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1094gitlab # [ 39.876664] postgres[1732]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1095gitlab # [ 39.877065] postgres[1732]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1096gitlab # [ 39.878164] postgres[1732]: attidentity AS identity,1097gitlab # [ 39.879282] postgres[1732]: attgenerated as attgenerated1098gitlab # [ 39.880404] postgres[1732]: FROM pg_attribute a1099gitlab # [ 39.881887] postgres[1732]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1100gitlab # [ 39.883056] postgres[1732]: LEFT JOIN pg_type t ON a.atttypid = t.oid1101gitlab # [ 39.884160] postgres[1732]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1102gitlab # [ 39.885297] postgres[1732]: WHERE a.attrelid = '"feature_gates"'::regclass1103gitlab # [ 39.886030] postgres[1732]: AND a.attnum > 0 AND NOT a.attisdropped1104gitlab # [ 39.887167] postgres[1732]: ORDER BY a.attnum1105gitlab # [ 39.888280] postgres[1732]:1106gitlab # [ 40.106714] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1370]: W, [2026-07-18T06:31:27.083610 #1370] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1107gitlab # [ 45.214485] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1788]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1108gitlab # [ 45.215412] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1788]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1109gitlab # [ 48.526662] gitlab-workhorse[1290]: time="2026-07-18T06:31:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1110gitlab # [ 58.527704] gitlab-workhorse[1290]: time="2026-07-18T06:31:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1111gitlab # [ 59.013953] postgres[1854]: [1854] ERROR: relation "table_without_models" does not exist1112gitlab # [ 59.015136] postgres[1854]: [1854] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1113gitlab # [ 68.538102] gitlab-workhorse[1290]: time="2026-07-18T06:31:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1114gitlab # [ 71.169224] systemd[1]: Finished gitlab-db-config.service.1115gitlab # [ 71.172915] systemd[1]: Started gitlab-sidekiq.service.1116gitlab # [ 71.177141] systemd[1]: Starting gitlab.service...1117gitlab # [ 71.499714] bundle[2021]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1118gitlab # [ 71.500725] bundle[2021]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1119gitlab # [ 71.501798] bundle[2021]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1120gitlab # [ 74.115713] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-07-18T06:32:01.076Z","message":"Starting cluster with 1 processes"}1121gitlab # [ 74.246898] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.206Z","pid":2021,"message":"* Enabling systemd notification integration"}1122gitlab # [ 74.248070] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.223Z","pid":2021,"message":"Puma starting in cluster mode..."}1123gitlab # [ 74.249197] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.223Z","pid":2021,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1124gitlab # [ 74.250412] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.223Z","pid":2021,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1125gitlab # [ 74.251473] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.223Z","pid":2021,"message":"* Min threads: 0"}1126gitlab # [ 74.252993] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.223Z","pid":2021,"message":"* Max threads: 4"}1127gitlab # [ 74.254199] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.224Z","pid":2021,"message":"* Environment: production"}1128gitlab # [ 74.255296] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.224Z","pid":2021,"message":"* Master PID: 2021"}1129gitlab # [ 74.256453] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.224Z","pid":2021,"message":"* Workers: 2"}1130gitlab # [ 74.257922] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.224Z","pid":2021,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1131gitlab # [ 74.259492] bundle[2021]: {"timestamp":"2026-07-18T06:32:01.224Z","pid":2021,"message":"* Preloading application"}1132gitlab # [ 78.547877] gitlab-workhorse[1290]: time="2026-07-18T06:32:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1133gitlab # [ 88.557771] gitlab-workhorse[1290]: time="2026-07-18T06:32:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1134gitlab # [ 96.226931] bundle[2021]: W, [2026-07-18T06:32:23.203748 #2021] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1135gitlab # [ 98.564417] gitlab-workhorse[1290]: time="2026-07-18T06:32:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1136gitlab # [ 98.706209] sidekiq-cluster[2051]: 2026-07-18T06:32:25.683Z pid=2051 tid=txn INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2051, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>429858816}1137gitlab # [ 98.876868] sidekiq-cluster[2051]: W, [2026-07-18T06:32:25.854248 #2051] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1138gitlab # [ 98.914791] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:25.892Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1139gitlab # [ 100.653671] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.630Z","message":"GitLab reliable fetch activated!"}1140gitlab # [ 100.653989] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.630Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1141gitlab # [ 100.692652] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.670Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1142gitlab # [ 100.702175] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.679Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1143gitlab # [ 100.710616] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.688Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1144gitlab # [ 100.714673] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.692Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1145gitlab # [ 100.725728] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.703Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1146gitlab # [ 100.733654] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.711Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1147gitlab # [ 100.741646] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.719Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1148gitlab # [ 100.745550] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.723Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1149gitlab # [ 100.753765] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.731Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1150gitlab # [ 100.761376] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.738Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1151gitlab # [ 100.769295] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.746Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1152gitlab # [ 100.776900] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.754Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1153gitlab # [ 100.783592] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.761Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1154gitlab # [ 100.790119] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.767Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1155gitlab # [ 100.796902] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.774Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1156gitlab # [ 100.805265] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.782Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1157gitlab # [ 100.812565] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.790Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1158gitlab # [ 100.821349] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.798Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1159gitlab # [ 100.829299] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.806Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1160gitlab # [ 100.837441] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.814Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1161gitlab # [ 100.845511] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.823Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1162gitlab # [ 100.853898] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.831Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1163gitlab # [ 100.861826] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.839Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1164gitlab # [ 100.869238] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.846Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1165gitlab # [ 100.876172] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.853Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1166gitlab # [ 100.883488] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.861Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1167gitlab # [ 100.889861] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.867Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1168gitlab # [ 100.895525] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.873Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1169gitlab # [ 100.901311] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.878Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1170gitlab # [ 100.907338] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.884Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1171gitlab # [ 100.913064] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.890Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1172gitlab # [ 100.917887] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.895Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1173gitlab # [ 100.924098] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.901Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1174gitlab # [ 100.929691] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.907Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1175gitlab # [ 100.935936] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.913Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1176gitlab # [ 100.941252] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.918Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1177gitlab # [ 100.946475] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.924Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1178gitlab # [ 100.951524] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.929Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1179gitlab # [ 100.957254] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.934Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1180gitlab # [ 100.961785] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.939Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1181gitlab # [ 100.966087] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.943Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1182gitlab # [ 100.970830] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.948Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1183gitlab # [ 100.975375] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.952Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1184gitlab # [ 100.980210] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.957Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1185gitlab # [ 100.984986] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.962Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1186gitlab # [ 100.990457] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.968Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1187gitlab # [ 100.995573] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.973Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1188gitlab # [ 101.000083] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.977Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1189gitlab # [ 101.004919] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.982Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1190gitlab # [ 101.010104] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.987Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1191gitlab # [ 101.015418] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.993Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1192gitlab # [ 101.020575] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:27.998Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1193gitlab # [ 101.026052] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.003Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1194gitlab # [ 101.038729] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.016Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1195gitlab # [ 101.043312] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.020Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1196gitlab # [ 101.047465] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.025Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1197gitlab # [ 101.051791] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.029Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1198gitlab # [ 101.055905] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.033Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1199gitlab # [ 101.060522] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.038Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1200gitlab # [ 101.064550] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.042Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1201gitlab # [ 101.071086] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.048Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1202gitlab # [ 101.073899] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.051Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1203gitlab # [ 101.077335] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.054Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1204gitlab # [ 101.081257] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.058Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1205gitlab # [ 101.085411] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.063Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1206gitlab # [ 101.090100] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.067Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1207gitlab # [ 101.094112] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.071Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1208gitlab # [ 101.097730] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.075Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1209gitlab # [ 101.101182] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.078Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1210gitlab # [ 101.104711] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.082Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1211gitlab # [ 101.110017] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.087Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1212gitlab # [ 101.112821] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.090Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1213gitlab # [ 101.116560] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.094Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1214gitlab # [ 101.120105] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.097Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1215gitlab # [ 101.125279] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.102Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1216gitlab # [ 101.146883] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.124Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1217gitlab # [ 101.150629] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.128Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1218gitlab # [ 101.154133] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.131Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1219gitlab # [ 101.159918] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.137Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1220gitlab # [ 101.163162] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.140Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1221gitlab # [ 101.166498] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.144Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1222gitlab # [ 101.171715] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.149Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1223gitlab # [ 101.178160] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.155Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1224gitlab # [ 101.185299] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.162Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1225gitlab # [ 101.190141] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.167Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1226gitlab # [ 101.196909] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.174Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1227gitlab # [ 101.201694] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.179Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1228gitlab # [ 101.208694] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.186Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1229gitlab # [ 101.213908] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.191Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1230gitlab # [ 101.220511] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.198Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1231gitlab # [ 101.225584] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.203Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1232gitlab # [ 101.230626] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.208Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1233gitlab # [ 101.236491] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.214Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1234gitlab # [ 101.241463] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.219Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1235gitlab # [ 101.247751] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.225Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1236gitlab # [ 101.253586] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.231Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1237gitlab # [ 101.261099] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.238Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1238gitlab # [ 101.266196] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.243Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1239gitlab # [ 101.272245] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.249Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1240gitlab # [ 101.292074] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.269Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1241gitlab # [ 101.294489] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.271Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1242gitlab # [ 101.296740] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.272Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1243gitlab # [ 101.299068] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.274Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1244gitlab # [ 101.301119] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.277Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1245gitlab # [ 101.303271] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.278Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1246gitlab # [ 101.305621] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.280Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1247gitlab # [ 101.307920] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.281Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1248gitlab # [ 101.310080] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.283Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1249gitlab # [ 101.312435] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.286Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1250gitlab # [ 101.314539] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:28.288Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1251gitlab # [ 108.574677] gitlab-workhorse[1290]: time="2026-07-18T06:32:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1252gitlab # [ 118.584743] gitlab-workhorse[1290]: time="2026-07-18T06:32:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1253gitlab # [ 121.233747] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:48.210Z","message":"Booted Rails 7.2.3 application in production environment"}1254gitlab # [ 121.238373] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:48.216Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1255gitlab # [ 121.242931] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:48.220Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1256gitlab # [ 121.247085] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:48.224Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1257gitlab # [ 121.253402] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:48.230Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1258gitlab # [ 121.950143] bundle[2021]: {"timestamp":"2026-07-18T06:32:48.927Z","pid":2021,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1259gitlab # [ 121.950467] bundle[2021]: {"timestamp":"2026-07-18T06:32:48.927Z","pid":2021,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1260gitlab # [ 121.952272] bundle[2021]: {"timestamp":"2026-07-18T06:32:48.927Z","pid":2021,"message":"! #\u003cThread:0x00007171fbdfd908 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1261gitlab # [ 121.953506] bundle[2021]: {"timestamp":"2026-07-18T06:32:48.927Z","pid":2021,"message":"! #\u003cThread:0x00007171fc020528 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1262gitlab # [ 121.954992] bundle[2021]: {"timestamp":"2026-07-18T06:32:48.927Z","pid":2021,"message":"Use Ctrl-C to stop"}1263gitlab # [ 121.982565] bundle[2021]: {"timestamp":"2026-07-18T06:32:48.927Z","pid":2021,"message":"Sending status to systemd every 1.0 sec"}1264gitlab # [ 122.361285] bundle[2021]: {"timestamp":"2026-07-18T06:32:49.337Z","pid":2021,"message":"- Worker 1 (PID: 2520) booted in 0.33s, phase: 0"}1265gitlab # [ 122.370114] bundle[2021]: {"timestamp":"2026-07-18T06:32:49.347Z","pid":2021,"message":"- Worker 0 (PID: 2516) booted in 0.36s, phase: 0"}1266gitlab # [ 122.378052] systemd[1]: Started gitlab.service.1267gitlab # [ 122.378247] systemd[1]: Reached target Common target for all GitLab services..1268gitlab # [ 122.381163] systemd[1]: Starting setup-gitlab.service...1269gitlab # [ 122.400742] setup-gitlab-start[2553]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1270gitlab # [ 122.499242] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:49.476Z","message":"Cleaning working queues"}1271gitlab # [ 122.503661] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:49.481Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c529f0000fe2443180423dd9","created_at":"2026-07-18T06:31:57.577Z","trace_propagation_headers":{"sentry-trace":"f354eac80f3f40f0b212fcc80da53567-8b95045cb1184d1c","baggage":"sentry-trace_id=f354eac80f3f40f0b212fcc80da53567,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"f01e427ff23e79b878f00e5c870caee6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-18T06:31:57.583Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c529f0000fe2443180423dd9: start","job_status":"start","queue_duration_s":51.897069,"scheduling_latency_s":51.897114}1272gitlab # [ 122.729451] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:32:49.706Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c529f0000fe2443180423dd9","created_at":"2026-07-18T06:31:57.577Z","trace_propagation_headers":{"sentry-trace":"f354eac80f3f40f0b212fcc80da53567-8b95045cb1184d1c","baggage":"sentry-trace_id=f354eac80f3f40f0b212fcc80da53567,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"f01e427ff23e79b878f00e5c870caee6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-18T06:31:57.583Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c529f0000fe2443180423dd9: done: 0.225744 sec","job_status":"done","queue_duration_s":51.897069,"scheduling_latency_s":51.897114,"redis_calls":26,"redis_duration_s":0.00814,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004344,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002295,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001501,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10822464899999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.225744,"completed_at":"2026-07-18T06:32:49.706Z","load_balancing_strategy":"primary","db_duration_s":0.079855,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1273gitlab # [ 124.492405] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:32:51 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 20701274gitlab # [ 124.496730] setup-gitlab-start[2579]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1275gitlab # [ 124.497087] setup-gitlab-start[2580]: ++ jq -r .access_token1276gitlab # [ 125.205898] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:32:52 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 6931277gitlab # [ 125.211859] setup-gitlab-start[2553]: + BEARER=cfbc5ced91d84ae4a8edb1c738f9822583f4eea4e2da432c09a18c7d991d1bb11278gitlab # [ 125.212620] setup-gitlab-start[2553]: + AUTH='Authorization: Bearer cfbc5ced91d84ae4a8edb1c738f9822583f4eea4e2da432c09a18c7d991d1bb1'1279gitlab # [ 125.213306] setup-gitlab-start[2553]: + curl -fs -X PUT -H 'Authorization: Bearer cfbc5ced91d84ae4a8edb1c738f9822583f4eea4e2da432c09a18c7d991d1bb1' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1280gitlab # [ 126.354489] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:32:53 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 11261281gitlab # [ 126.357254] setup-gitlab-start[2553]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1282gitlab # [ 126.371890] runuser[2600]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1283gitlab # [ 126.687099] setup-gitlab-start[2601]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1284gitlab # [ 126.689341] setup-gitlab-start[2601]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1285gitlab # [ 126.691295] setup-gitlab-start[2601]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1286gitlab # [ 127.214205] setup-gitlab-start[2601]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1287gitlab # [ 127.214827] setup-gitlab-start[2601]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1288gitlab # [ 127.217856] setup-gitlab-start[2601]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1289gitlab # [ 134.406860] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:01.383Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1be2743e6ae5ff2cca3cbd9e","created_at":"2026-07-18T06:33:01.351Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed41c9214b3a6782523355918658b02c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.380Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1be2743e6ae5ff2cca3cbd9e: start","job_status":"start","queue_duration_s":0.002876,"scheduling_latency_s":0.002924}1290gitlab # [ 135.030398] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1be2743e6ae5ff2cca3cbd9e","created_at":"2026-07-18T06:33:01.351Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed41c9214b3a6782523355918658b02c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.380Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1be2743e6ae5ff2cca3cbd9e: done: 0.624495 sec","job_status":"done","queue_duration_s":0.002876,"scheduling_latency_s":0.002924,"redis_calls":574,"redis_duration_s":0.339923,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006722,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.330868,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.281592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000293613,"duration_s":0.624495,"completed_at":"2026-07-18T06:33:02.007Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 135.068354] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d7db18b10e36da4cadb1d97b","created_at":"2026-07-18T06:33:01.416Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebe33e102b14758b201e763671c237a4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.424Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d7db18b10e36da4cadb1d97b: start","job_status":"start","queue_duration_s":0.620694,"scheduling_latency_s":0.620716}1292gitlab # [ 135.313709] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d7db18b10e36da4cadb1d97b","created_at":"2026-07-18T06:33:01.416Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebe33e102b14758b201e763671c237a4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.424Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d7db18b10e36da4cadb1d97b: done: 0.244921 sec","job_status":"done","queue_duration_s":0.620694,"scheduling_latency_s":0.620716,"redis_calls":12,"redis_duration_s":0.003597,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001538,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001027505,"exclusive_lock_hold_duration_s":0.18515062599999998,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.244921,"completed_at":"2026-07-18T06:33:02.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1293gitlab # [ 135.384629] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"00b3d4cdc205019545d9e846","created_at":"2026-07-18T06:33:01.442Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0274338e4fed9a2b44a811b0e8bc199","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.456Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-00b3d4cdc205019545d9e846: start","job_status":"start","queue_duration_s":0.904456,"scheduling_latency_s":0.904504}1294gitlab # [ 135.403447] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.380Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1295gitlab # [ 135.417806] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"00b3d4cdc205019545d9e846","created_at":"2026-07-18T06:33:01.442Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0274338e4fed9a2b44a811b0e8bc199","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.456Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-00b3d4cdc205019545d9e846: done: 0.032978 sec","job_status":"done","queue_duration_s":0.904456,"scheduling_latency_s":0.904504,"redis_calls":10,"redis_duration_s":0.005256,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003963,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000524927,"duration_s":0.032978,"completed_at":"2026-07-18T06:33:02.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 135.455817] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4331f000524aaceba2e59281","created_at":"2026-07-18T06:33:01.467Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67093bf0c96fac5932a7934c6d20233d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.477Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4331f000524aaceba2e59281: start","job_status":"start","queue_duration_s":0.955467,"scheduling_latency_s":0.955506}1297gitlab # [ 135.497632] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4331f000524aaceba2e59281","created_at":"2026-07-18T06:33:01.467Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67093bf0c96fac5932a7934c6d20233d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.477Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4331f000524aaceba2e59281: done: 0.041605 sec","job_status":"done","queue_duration_s":0.955467,"scheduling_latency_s":0.955506,"redis_calls":12,"redis_duration_s":0.002473,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000799,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000420445,"duration_s":0.041605,"completed_at":"2026-07-18T06:33:02.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 135.537064] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.514Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2825991774d513682e02e515","created_at":"2026-07-18T06:33:01.491Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9355d22265f175279026a87e6f62b02","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.501Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2825991774d513682e02e515: start","job_status":"start","queue_duration_s":1.012961,"scheduling_latency_s":1.013001}1299gitlab # [ 135.573205] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2825991774d513682e02e515","created_at":"2026-07-18T06:33:01.491Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9355d22265f175279026a87e6f62b02","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.501Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2825991774d513682e02e515: done: 0.035879 sec","job_status":"done","queue_duration_s":1.012961,"scheduling_latency_s":1.013001,"redis_calls":12,"redis_duration_s":0.003793,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000619,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000722439,"exclusive_lock_hold_duration_s":0.007095036,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.035879,"completed_at":"2026-07-18T06:33:02.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003747,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 135.612548] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"84ce4701beb6a9177deebd99","created_at":"2026-07-18T06:33:01.511Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.521Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-84ce4701beb6a9177deebd99: start","job_status":"start","queue_duration_s":1.067937,"scheduling_latency_s":1.067975}1301gitlab # [ 135.659146] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"84ce4701beb6a9177deebd99","created_at":"2026-07-18T06:33:01.511Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.521Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-84ce4701beb6a9177deebd99: done: 0.046751 sec","job_status":"done","queue_duration_s":1.067937,"scheduling_latency_s":1.067975,"redis_calls":19,"redis_duration_s":0.005547,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001311,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000399,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002912,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424355,"duration_s":0.046751,"completed_at":"2026-07-18T06:33:02.636Z","load_balancing_strategy":"primary","db_duration_s":0.002869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 135.697897] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.675Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"39fd30abba65d828a4e5bd50","created_at":"2026-07-18T06:33:01.538Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1dc7959e7d10ee58f8b5ba15034b79bf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.547Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-39fd30abba65d828a4e5bd50: start","job_status":"start","queue_duration_s":1.127455,"scheduling_latency_s":1.127491}1303gitlab # [ 135.742113] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"39fd30abba65d828a4e5bd50","created_at":"2026-07-18T06:33:01.538Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1dc7959e7d10ee58f8b5ba15034b79bf","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.547Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-39fd30abba65d828a4e5bd50: done: 0.042378 sec","job_status":"done","queue_duration_s":1.127455,"scheduling_latency_s":1.127491,"redis_calls":16,"redis_duration_s":0.004813,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002542,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001606,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038273,"duration_s":0.042378,"completed_at":"2026-07-18T06:33:02.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 135.780136] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a87dbde9e003a13bc550231b","created_at":"2026-07-18T06:33:01.560Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"293e78afc869812fd734f601868a0aec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-18T06:33:01.569Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a87dbde9e003a13bc550231b: start","job_status":"start","queue_duration_s":1.149562,"scheduling_latency_s":1.149586}1305gitlab # [ 135.791756] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a87dbde9e003a13bc550231b","created_at":"2026-07-18T06:33:01.560Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"293e78afc869812fd734f601868a0aec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-18T06:33:01.569Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a87dbde9e003a13bc550231b: done: 0.022384 sec","job_status":"done","queue_duration_s":1.149562,"scheduling_latency_s":1.149586,"redis_calls":10,"redis_duration_s":0.001434,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199187,"duration_s":0.022384,"completed_at":"2026-07-18T06:33:02.741Z","load_balancing_strategy":"primary","db_duration_s":0.003307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 135.827987] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"467a77b6f23b44562ad47ce7","created_at":"2026-07-18T06:33:01.579Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1567e3a811c3f150d391f896e1de45ae","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.588Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-467a77b6f23b44562ad47ce7: start","job_status":"start","queue_duration_s":1.154301,"scheduling_latency_s":1.154314}1307gitlab # [ 135.840337] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"467a77b6f23b44562ad47ce7","created_at":"2026-07-18T06:33:01.579Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1567e3a811c3f150d391f896e1de45ae","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.588Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-467a77b6f23b44562ad47ce7: done: 0.015438 sec","job_status":"done","queue_duration_s":1.154301,"scheduling_latency_s":1.154314,"redis_calls":12,"redis_duration_s":0.001639,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000284,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000419327,"exclusive_lock_hold_duration_s":0.00393821,"duration_s":0.015438,"completed_at":"2026-07-18T06:33:02.757Z","load_balancing_strategy":"primary","db_duration_s":0.001811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 135.840971] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"385af3a9243259c1c55f3877","created_at":"2026-07-18T06:33:01.600Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eed1b4053334919edf829630bebac24a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.610Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-385af3a9243259c1c55f3877: start","job_status":"start","queue_duration_s":1.14871,"scheduling_latency_s":1.148724}1309gitlab # [ 135.930153] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"385af3a9243259c1c55f3877","created_at":"2026-07-18T06:33:01.600Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eed1b4053334919edf829630bebac24a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.610Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-385af3a9243259c1c55f3877: done: 0.148695 sec","job_status":"done","queue_duration_s":1.14871,"scheduling_latency_s":1.148724,"redis_calls":14,"redis_duration_s":0.002512,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00031261,"exclusive_lock_hold_duration_s":0.12454432,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.148695,"completed_at":"2026-07-18T06:33:02.907Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 135.969950] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"39b06dc169d76a2d2cf04b22","created_at":"2026-07-18T06:33:01.628Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b82ecd95f663784af4f4f00fcda6b5c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.636Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-39b06dc169d76a2d2cf04b22: start","job_status":"start","queue_duration_s":1.311093,"scheduling_latency_s":1.311117}1311gitlab # [ 136.000578] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:02.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"39b06dc169d76a2d2cf04b22","created_at":"2026-07-18T06:33:01.628Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1b82ecd95f663784af4f4f00fcda6b5c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.636Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-39b06dc169d76a2d2cf04b22: done: 0.030684 sec","job_status":"done","queue_duration_s":1.311093,"scheduling_latency_s":1.311117,"redis_calls":10,"redis_duration_s":0.002005,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200584,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.030684,"completed_at":"2026-07-18T06:33:02.977Z","load_balancing_strategy":"primary","db_duration_s":0.013412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 136.038391] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"310ae158ccaba7039d9f4eef","created_at":"2026-07-18T06:33:01.651Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae70822697f8aead883ac8eec9b47920","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.660Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-310ae158ccaba7039d9f4eef: start","job_status":"start","queue_duration_s":1.354833,"scheduling_latency_s":1.354857}1313gitlab # [ 136.082782] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.059Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"310ae158ccaba7039d9f4eef","created_at":"2026-07-18T06:33:01.651Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae70822697f8aead883ac8eec9b47920","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.660Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-310ae158ccaba7039d9f4eef: done: 0.043924 sec","job_status":"done","queue_duration_s":1.354833,"scheduling_latency_s":1.354857,"redis_calls":13,"redis_duration_s":0.002674,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001552,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011422681,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.043924,"completed_at":"2026-07-18T06:33:03.059Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002852,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1314gitlab # [ 136.085326] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6846cd7a0cf9369b4d61a2fd","created_at":"2026-07-18T06:33:01.668Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5daa08d83da58a32df7938d00da67ea","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.676Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6846cd7a0cf9369b4d61a2fd: start","job_status":"start","queue_duration_s":1.386297,"scheduling_latency_s":1.386327}1315gitlab # [ 136.156153] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6846cd7a0cf9369b4d61a2fd","created_at":"2026-07-18T06:33:01.668Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b5daa08d83da58a32df7938d00da67ea","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.676Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6846cd7a0cf9369b4d61a2fd: done: 0.070724 sec","job_status":"done","queue_duration_s":1.386297,"scheduling_latency_s":1.386327,"redis_calls":10,"redis_duration_s":0.002505,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000347251,"duration_s":0.070724,"completed_at":"2026-07-18T06:33:03.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04308,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 136.193879] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"bb429f6260825f431ad39ef0","created_at":"2026-07-18T06:33:01.683Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.689Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-bb429f6260825f431ad39ef0: start","job_status":"start","queue_duration_s":1.481388,"scheduling_latency_s":1.481414}1317gitlab # [ 136.275482] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"bb429f6260825f431ad39ef0","created_at":"2026-07-18T06:33:01.683Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.689Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-bb429f6260825f431ad39ef0: done: 0.081351 sec","job_status":"done","queue_duration_s":1.481388,"scheduling_latency_s":1.481414,"redis_calls":28,"redis_duration_s":0.01108,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001581,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000539,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.004615,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000392508,"duration_s":0.081351,"completed_at":"2026-07-18T06:33:03.252Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 136.277297] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"0ab47e1c1cee2646075084de","created_at":"2026-07-18T06:33:01.723Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27c064b134f07d0571a2b138834f3c02","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.731Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-0ab47e1c1cee2646075084de: start","job_status":"start","queue_duration_s":1.523627,"scheduling_latency_s":1.523655}1319gitlab # [ 136.338903] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"0ab47e1c1cee2646075084de","created_at":"2026-07-18T06:33:01.723Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27c064b134f07d0571a2b138834f3c02","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.731Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-0ab47e1c1cee2646075084de: done: 0.021405 sec","job_status":"done","queue_duration_s":1.523627,"scheduling_latency_s":1.523655,"redis_calls":12,"redis_duration_s":0.002826,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000390273,"exclusive_lock_hold_duration_s":0.007899328,"duration_s":0.021405,"completed_at":"2026-07-18T06:33:03.275Z","load_balancing_strategy":"primary","db_duration_s":0.00469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 136.347241] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.277Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1535a86342d0f89d42c87997","created_at":"2026-07-18T06:33:01.737Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c9c7784ae687267e93940aab4cc81a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-18T06:33:01.744Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1535a86342d0f89d42c87997: start","job_status":"start","queue_duration_s":1.532519,"scheduling_latency_s":1.532531}1321gitlab # [ 136.360335] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.285Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1322gitlab # [ 136.363907] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1535a86342d0f89d42c87997","created_at":"2026-07-18T06:33:01.737Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c9c7784ae687267e93940aab4cc81a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-18T06:33:01.744Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1535a86342d0f89d42c87997: done: 0.012798 sec","job_status":"done","queue_duration_s":1.532519,"scheduling_latency_s":1.532531,"redis_calls":10,"redis_duration_s":0.0019950000000000002,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000273219,"duration_s":0.012798,"completed_at":"2026-07-18T06:33:03.289Z","load_balancing_strategy":"primary","db_duration_s":0.000417,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 136.422531] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"cee4a1fd788e09c94fafc407","created_at":"2026-07-18T06:33:01.765Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4c6be2a6ca8095ed3acf6cccb623d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-18T06:33:01.771Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-cee4a1fd788e09c94fafc407: start","job_status":"start","queue_duration_s":1.519503,"scheduling_latency_s":1.519516}1324gitlab # [ 136.423734] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.298Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1325gitlab # [ 136.424807] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"cee4a1fd788e09c94fafc407","created_at":"2026-07-18T06:33:01.765Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4c6be2a6ca8095ed3acf6cccb623d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-18T06:33:01.771Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-cee4a1fd788e09c94fafc407: done: 0.018564 sec","job_status":"done","queue_duration_s":1.519503,"scheduling_latency_s":1.519516,"redis_calls":10,"redis_duration_s":0.002343,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000344,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000615,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000740597,"duration_s":0.018564,"completed_at":"2026-07-18T06:33:03.309Z","load_balancing_strategy":"primary","db_duration_s":0.000526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 136.454683] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c7a0432e1410602e4486008c","created_at":"2026-07-18T06:33:01.778Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c54b03843bc050dcc6303b758e82c8e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.784Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c7a0432e1410602e4486008c: start","job_status":"start","queue_duration_s":1.527148,"scheduling_latency_s":1.527164}1327gitlab # [ 136.506030] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.321Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1328gitlab # [ 136.506773] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c7a0432e1410602e4486008c","created_at":"2026-07-18T06:33:01.778Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c54b03843bc050dcc6303b758e82c8e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.784Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c7a0432e1410602e4486008c: done: 0.01517 sec","job_status":"done","queue_duration_s":1.527148,"scheduling_latency_s":1.527164,"redis_calls":10,"redis_duration_s":0.00152,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210362,"duration_s":0.01517,"completed_at":"2026-07-18T06:33:03.326Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 136.509574] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ce570f2c3ab36c3cda2c4279","created_at":"2026-07-18T06:33:01.793Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a4fda7c3123e543a2335583193227a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.804Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ce570f2c3ab36c3cda2c4279: start","job_status":"start","queue_duration_s":1.523616,"scheduling_latency_s":1.523656}1330gitlab # [ 136.555055] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ce570f2c3ab36c3cda2c4279","created_at":"2026-07-18T06:33:01.793Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a4fda7c3123e543a2335583193227a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.804Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ce570f2c3ab36c3cda2c4279: done: 0.030431 sec","job_status":"done","queue_duration_s":1.523616,"scheduling_latency_s":1.523656,"redis_calls":12,"redis_duration_s":0.001654,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000631,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000306463,"exclusive_lock_hold_duration_s":0.017340472000000003,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.030431,"completed_at":"2026-07-18T06:33:03.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 136.563120] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"58f8bb8565a8cea19ee0e984","created_at":"2026-07-18T06:33:01.816Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.822Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-58f8bb8565a8cea19ee0e984: start","job_status":"start","queue_duration_s":1.536225,"scheduling_latency_s":1.536239}1332gitlab # [ 136.606526] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.425Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"58f8bb8565a8cea19ee0e984","created_at":"2026-07-18T06:33:01.816Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.822Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-58f8bb8565a8cea19ee0e984: done: 0.066632 sec","job_status":"done","queue_duration_s":1.536225,"scheduling_latency_s":1.536239,"redis_calls":71,"redis_duration_s":0.013769,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001592,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000489,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.011282,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214832,"duration_s":0.066632,"completed_at":"2026-07-18T06:33:03.425Z","load_balancing_strategy":"primary","db_duration_s":0.001008,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 136.647712] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ec2f9a4ec0fba88f1e526f68","created_at":"2026-07-18T06:33:01.828Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fc293218e5d74f83fd0686904d3e779","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.834Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ec2f9a4ec0fba88f1e526f68: start","job_status":"start","queue_duration_s":1.591723,"scheduling_latency_s":1.591737}1334gitlab # [ 136.648604] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.437Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1335gitlab # [ 136.649204] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ec2f9a4ec0fba88f1e526f68","created_at":"2026-07-18T06:33:01.828Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fc293218e5d74f83fd0686904d3e779","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.834Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ec2f9a4ec0fba88f1e526f68: done: 0.014976 sec","job_status":"done","queue_duration_s":1.591723,"scheduling_latency_s":1.591737,"redis_calls":10,"redis_duration_s":0.001251,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000190806,"duration_s":0.014976,"completed_at":"2026-07-18T06:33:03.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003043,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 136.693633] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"41b6ccdb230bfa8d82f300e0","created_at":"2026-07-18T06:33:01.840Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdd88a37400c419fd2bb420bb4d1b9cd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.846Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-41b6ccdb230bfa8d82f300e0: start","job_status":"start","queue_duration_s":1.596113,"scheduling_latency_s":1.596125}1337gitlab # [ 136.694080] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.449Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1338gitlab # [ 136.694599] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"41b6ccdb230bfa8d82f300e0","created_at":"2026-07-18T06:33:01.840Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdd88a37400c419fd2bb420bb4d1b9cd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.846Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-41b6ccdb230bfa8d82f300e0: done: 0.011181 sec","job_status":"done","queue_duration_s":1.596113,"scheduling_latency_s":1.596125,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000219581,"duration_s":0.011181,"completed_at":"2026-07-18T06:33:03.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000232,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 136.739613] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d4e38d069dc6ae4fbaaf6e4f","created_at":"2026-07-18T06:33:01.855Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2befc44925ceaa3af2559afec5d6d000","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.860Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d4e38d069dc6ae4fbaaf6e4f: start","job_status":"start","queue_duration_s":1.594556,"scheduling_latency_s":1.594569}1340gitlab # [ 136.740579] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d4e38d069dc6ae4fbaaf6e4f","created_at":"2026-07-18T06:33:01.855Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4c10e2894da24a76","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2befc44925ceaa3af2559afec5d6d000","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:01.860Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d4e38d069dc6ae4fbaaf6e4f: done: 0.011513 sec","job_status":"done","queue_duration_s":1.594556,"scheduling_latency_s":1.594569,"redis_calls":10,"redis_duration_s":0.001212,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000186057,"duration_s":0.011513,"completed_at":"2026-07-18T06:33:03.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 136.785674] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.466Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0eb5ac6c0bcb958658a5d10","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0eb5ac6c0bcb958658a5d10: start","job_status":"start","queue_duration_s":0.839464,"scheduling_latency_s":0.839477}1342gitlab # [ 136.788973] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.476Z","message":"No existing merge request to be cleaned up."}1343gitlab # [ 136.790827] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.483Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0eb5ac6c0bcb958658a5d10","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0eb5ac6c0bcb958658a5d10: done: 0.016823 sec","job_status":"done","queue_duration_s":0.839464,"scheduling_latency_s":0.839477,"redis_calls":13,"redis_duration_s":0.00181,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00024,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021567,"duration_s":0.016823,"completed_at":"2026-07-18T06:33:03.483Z","load_balancing_strategy":"primary","db_duration_s":0.002072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 136.869975] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.484Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d556c5fb9e62d0d4b96dd016","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d556c5fb9e62d0d4b96dd016: start","job_status":"start","queue_duration_s":0.857166,"scheduling_latency_s":0.857179}1345gitlab # [ 136.882514] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.493Z","message":"No existing merge request to be cleaned up."}1346gitlab # [ 136.882814] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.498Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d556c5fb9e62d0d4b96dd016","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d556c5fb9e62d0d4b96dd016: done: 0.013853 sec","job_status":"done","queue_duration_s":0.857166,"scheduling_latency_s":0.857179,"redis_calls":10,"redis_duration_s":0.00135,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000807,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013853,"completed_at":"2026-07-18T06:33:03.498Z","load_balancing_strategy":"primary","db_duration_s":0.005219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 136.883807] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.499Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2e328b8658da2f7537c1b158","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2e328b8658da2f7537c1b158: start","job_status":"start","queue_duration_s":0.871971,"scheduling_latency_s":0.871985}1348gitlab # [ 136.889646] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.505Z","message":"No existing merge request to be cleaned up."}1349gitlab # [ 136.891231] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.509Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2e328b8658da2f7537c1b158","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2e328b8658da2f7537c1b158: done: 0.010371 sec","job_status":"done","queue_duration_s":0.871971,"scheduling_latency_s":0.871985,"redis_calls":10,"redis_duration_s":0.0012690000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010371,"completed_at":"2026-07-18T06:33:03.509Z","load_balancing_strategy":"primary","db_duration_s":0.000933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 136.943305] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.510Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cf57742a8cd642d80da8537e","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cf57742a8cd642d80da8537e: start","job_status":"start","queue_duration_s":0.883102,"scheduling_latency_s":0.883115}1351gitlab # [ 136.944029] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.516Z","message":"No existing merge request to be cleaned up."}1352gitlab # [ 136.944528] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.520Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cf57742a8cd642d80da8537e","created_at":"2026-07-18T06:33:02.611Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-1a0107c839f44203","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d3cd63648c4e96e31ccb4e7897c101b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:02.627Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cf57742a8cd642d80da8537e: done: 0.010194 sec","job_status":"done","queue_duration_s":0.883102,"scheduling_latency_s":0.883115,"redis_calls":10,"redis_duration_s":0.0012569999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010194,"completed_at":"2026-07-18T06:33:03.520Z","load_balancing_strategy":"primary","db_duration_s":0.000921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 136.994501] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"104d23b6241dc4d2f0cb7fa9","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-104d23b6241dc4d2f0cb7fa9: start","job_status":"start","queue_duration_s":0.279141,"scheduling_latency_s":0.279154}1354gitlab # [ 136.995181] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"104d23b6241dc4d2f0cb7fa9","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-104d23b6241dc4d2f0cb7fa9: done: 0.029421 sec","job_status":"done","queue_duration_s":0.279141,"scheduling_latency_s":0.279154,"redis_calls":16,"redis_duration_s":0.002171,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021986,"duration_s":0.029421,"completed_at":"2026-07-18T06:33:03.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 137.033305] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"5519153032df2b74ef580a19","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-5519153032df2b74ef580a19: start","job_status":"start","queue_duration_s":0.30943,"scheduling_latency_s":0.309444}1356gitlab # [ 137.034054] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"5519153032df2b74ef580a19","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-5519153032df2b74ef580a19: done: 0.010224 sec","job_status":"done","queue_duration_s":0.30943,"scheduling_latency_s":0.309444,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010224,"completed_at":"2026-07-18T06:33:03.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 137.073258] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"594c8d579f096e2f78fe9dfd","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-594c8d579f096e2f78fe9dfd: start","job_status":"start","queue_duration_s":0.320602,"scheduling_latency_s":0.320615}1358gitlab # [ 137.074639] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"594c8d579f096e2f78fe9dfd","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-594c8d579f096e2f78fe9dfd: done: 0.009654 sec","job_status":"done","queue_duration_s":0.320602,"scheduling_latency_s":0.320615,"redis_calls":13,"redis_duration_s":0.0015539999999999998,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009654,"completed_at":"2026-07-18T06:33:03.572Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000473,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 137.125327] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"424777f0b7e55eac4de119a5","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-424777f0b7e55eac4de119a5: start","job_status":"start","queue_duration_s":0.331381,"scheduling_latency_s":0.331394}1360gitlab # [ 137.126051] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"424777f0b7e55eac4de119a5","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-424777f0b7e55eac4de119a5: done: 0.009698 sec","job_status":"done","queue_duration_s":0.331381,"scheduling_latency_s":0.331394,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009698,"completed_at":"2026-07-18T06:33:03.583Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 137.166640] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"35af7a032f6dca24169b32e2","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-35af7a032f6dca24169b32e2: start","job_status":"start","queue_duration_s":0.341952,"scheduling_latency_s":0.341966}1362gitlab # [ 137.167440] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"35af7a032f6dca24169b32e2","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-35af7a032f6dca24169b32e2: done: 0.009829 sec","job_status":"done","queue_duration_s":0.341952,"scheduling_latency_s":0.341966,"redis_calls":13,"redis_duration_s":0.001603,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009829,"completed_at":"2026-07-18T06:33:03.594Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 137.224307] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9fa6c968bfb2528e62517ce4","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9fa6c968bfb2528e62517ce4: start","job_status":"start","queue_duration_s":0.352735,"scheduling_latency_s":0.352755}1364gitlab # [ 137.224995] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9fa6c968bfb2528e62517ce4","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9fa6c968bfb2528e62517ce4: done: 0.009708 sec","job_status":"done","queue_duration_s":0.352735,"scheduling_latency_s":0.352755,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009708,"completed_at":"2026-07-18T06:33:03.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 137.264467] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c88577b298397d9327a43e9c","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c88577b298397d9327a43e9c: start","job_status":"start","queue_duration_s":0.363373,"scheduling_latency_s":0.363386}1366gitlab # [ 137.265163] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c88577b298397d9327a43e9c","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c88577b298397d9327a43e9c: done: 0.009575 sec","job_status":"done","queue_duration_s":0.363373,"scheduling_latency_s":0.363386,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009575,"completed_at":"2026-07-18T06:33:03.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 137.315727] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"41877934fbcaf622626334c0","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-41877934fbcaf622626334c0: start","job_status":"start","queue_duration_s":0.373905,"scheduling_latency_s":0.37392}1368gitlab # [ 137.316617] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"41877934fbcaf622626334c0","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-41877934fbcaf622626334c0: done: 0.009563 sec","job_status":"done","queue_duration_s":0.373905,"scheduling_latency_s":0.37392,"redis_calls":13,"redis_duration_s":0.001537,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009563,"completed_at":"2026-07-18T06:33:03.625Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 137.363924] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"014797ff900fc6f22371ce49","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-014797ff900fc6f22371ce49: start","job_status":"start","queue_duration_s":0.384382,"scheduling_latency_s":0.384396}1370gitlab # [ 137.364142] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"014797ff900fc6f22371ce49","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-014797ff900fc6f22371ce49: done: 0.01005 sec","job_status":"done","queue_duration_s":0.384382,"scheduling_latency_s":0.384396,"redis_calls":13,"redis_duration_s":0.0016550000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01005,"completed_at":"2026-07-18T06:33:03.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000424,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 137.412919] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"107a1aee176a6b3db9eb1c41","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-107a1aee176a6b3db9eb1c41: start","job_status":"start","queue_duration_s":0.395367,"scheduling_latency_s":0.395381}1372gitlab # [ 137.413669] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"107a1aee176a6b3db9eb1c41","created_at":"2026-07-18T06:33:03.217Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-87040cbfc4c445f1","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"057fceac420fd69eb60441d34f1ec4d7","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.242Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-107a1aee176a6b3db9eb1c41: done: 0.009712 sec","job_status":"done","queue_duration_s":0.395367,"scheduling_latency_s":0.395381,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009712,"completed_at":"2026-07-18T06:33:03.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 137.460280] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53079021115baa2765fd2ff3","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53079021115baa2765fd2ff3: start","job_status":"start","queue_duration_s":0.228744,"scheduling_latency_s":0.228757}1374gitlab # [ 137.460906] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53079021115baa2765fd2ff3","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53079021115baa2765fd2ff3: done: 0.013706 sec","job_status":"done","queue_duration_s":0.228744,"scheduling_latency_s":0.228757,"redis_calls":16,"redis_duration_s":0.001915,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197232,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013706,"completed_at":"2026-07-18T06:33:03.662Z","load_balancing_strategy":"primary","db_duration_s":0.000283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 137.474679] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85283d28f96567eaece0a13b","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85283d28f96567eaece0a13b: start","job_status":"start","queue_duration_s":0.243406,"scheduling_latency_s":0.243419}1376gitlab # [ 137.521116] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85283d28f96567eaece0a13b","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85283d28f96567eaece0a13b: done: 0.009532 sec","job_status":"done","queue_duration_s":0.243406,"scheduling_latency_s":0.243419,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009532,"completed_at":"2026-07-18T06:33:03.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 137.522099] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3638968d97c1310688a8e068","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3638968d97c1310688a8e068: start","job_status":"start","queue_duration_s":0.254276,"scheduling_latency_s":0.25429}1378gitlab # [ 137.529921] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3638968d97c1310688a8e068","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3638968d97c1310688a8e068: done: 0.009291 sec","job_status":"done","queue_duration_s":0.254276,"scheduling_latency_s":0.25429,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009291,"completed_at":"2026-07-18T06:33:03.683Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 137.570960] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c22804e1a81a963e06d42944","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c22804e1a81a963e06d42944: start","job_status":"start","queue_duration_s":0.264531,"scheduling_latency_s":0.264544}1380gitlab # [ 137.574796] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c22804e1a81a963e06d42944","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c22804e1a81a963e06d42944: done: 0.0092 sec","job_status":"done","queue_duration_s":0.264531,"scheduling_latency_s":0.264544,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0092,"completed_at":"2026-07-18T06:33:03.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 137.623473] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b5bfe509ca4e7a8c5551969","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b5bfe509ca4e7a8c5551969: start","job_status":"start","queue_duration_s":0.274526,"scheduling_latency_s":0.274539}1382gitlab # [ 137.624100] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b5bfe509ca4e7a8c5551969","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b5bfe509ca4e7a8c5551969: done: 0.009959 sec","job_status":"done","queue_duration_s":0.274526,"scheduling_latency_s":0.274539,"redis_calls":13,"redis_duration_s":0.002087,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001468,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009959,"completed_at":"2026-07-18T06:33:03.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 137.665666] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a72bf11c6766bf8408fc2f51","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a72bf11c6766bf8408fc2f51: start","job_status":"start","queue_duration_s":0.285464,"scheduling_latency_s":0.285477}1384gitlab # [ 137.666413] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a72bf11c6766bf8408fc2f51","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a72bf11c6766bf8408fc2f51: done: 0.010253 sec","job_status":"done","queue_duration_s":0.285464,"scheduling_latency_s":0.285477,"redis_calls":13,"redis_duration_s":0.0021739999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001467,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010253,"completed_at":"2026-07-18T06:33:03.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 137.704250] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a79dafa28086c33c3be43dc7","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a79dafa28086c33c3be43dc7: start","job_status":"start","queue_duration_s":0.29662,"scheduling_latency_s":0.296634}1386gitlab # [ 137.708948] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a79dafa28086c33c3be43dc7","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a79dafa28086c33c3be43dc7: done: 0.009898 sec","job_status":"done","queue_duration_s":0.29662,"scheduling_latency_s":0.296634,"redis_calls":13,"redis_duration_s":0.001835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009898,"completed_at":"2026-07-18T06:33:03.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 137.763440] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1974786606f81dd5f160b70e","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1974786606f81dd5f160b70e: start","job_status":"start","queue_duration_s":0.307518,"scheduling_latency_s":0.307531}1388gitlab # [ 137.764029] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1974786606f81dd5f160b70e","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1974786606f81dd5f160b70e: done: 0.010246 sec","job_status":"done","queue_duration_s":0.307518,"scheduling_latency_s":0.307531,"redis_calls":13,"redis_duration_s":0.002183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001508,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010246,"completed_at":"2026-07-18T06:33:03.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 137.803600] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ced08ebaa12df68af4d810","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ced08ebaa12df68af4d810: start","job_status":"start","queue_duration_s":0.318708,"scheduling_latency_s":0.31872}1390gitlab # [ 137.804328] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ced08ebaa12df68af4d810","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ced08ebaa12df68af4d810: done: 0.010677 sec","job_status":"done","queue_duration_s":0.318708,"scheduling_latency_s":0.31872,"redis_calls":13,"redis_duration_s":0.002663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010677,"completed_at":"2026-07-18T06:33:03.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 137.858076] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d945005ea9f93e5f98769196","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d945005ea9f93e5f98769196: start","job_status":"start","queue_duration_s":0.330429,"scheduling_latency_s":0.330442}1392gitlab # [ 137.858787] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d945005ea9f93e5f98769196","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d945005ea9f93e5f98769196: done: 0.010444 sec","job_status":"done","queue_duration_s":0.330429,"scheduling_latency_s":0.330442,"redis_calls":13,"redis_duration_s":0.002638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001814,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010444,"completed_at":"2026-07-18T06:33:03.760Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 137.911152] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6178e4a6c35e9d01a59cef26","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6178e4a6c35e9d01a59cef26: start","job_status":"start","queue_duration_s":0.342034,"scheduling_latency_s":0.342048}1394gitlab # [ 137.911865] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6178e4a6c35e9d01a59cef26","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6178e4a6c35e9d01a59cef26: done: 0.009763 sec","job_status":"done","queue_duration_s":0.342034,"scheduling_latency_s":0.342048,"redis_calls":13,"redis_duration_s":0.001915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009763,"completed_at":"2026-07-18T06:33:03.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 137.955964] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"559e5a6d74fdeb82eb5f1143","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-559e5a6d74fdeb82eb5f1143: start","job_status":"start","queue_duration_s":0.352912,"scheduling_latency_s":0.352924}1396gitlab # [ 137.956583] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"559e5a6d74fdeb82eb5f1143","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-559e5a6d74fdeb82eb5f1143: done: 0.010261 sec","job_status":"done","queue_duration_s":0.352912,"scheduling_latency_s":0.352924,"redis_calls":13,"redis_duration_s":0.002452,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001688,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010261,"completed_at":"2026-07-18T06:33:03.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 137.994087] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d60114e7c9032afbbd19ff61","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d60114e7c9032afbbd19ff61: start","job_status":"start","queue_duration_s":0.364473,"scheduling_latency_s":0.364487}1398gitlab # [ 137.994652] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d60114e7c9032afbbd19ff61","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d60114e7c9032afbbd19ff61: done: 0.010433 sec","job_status":"done","queue_duration_s":0.364473,"scheduling_latency_s":0.364487,"redis_calls":13,"redis_duration_s":0.002511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001734,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010433,"completed_at":"2026-07-18T06:33:03.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 138.043985] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ede1940704c1bf11bd59114","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ede1940704c1bf11bd59114: start","job_status":"start","queue_duration_s":0.376097,"scheduling_latency_s":0.376111}1400gitlab # [ 138.044716] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ede1940704c1bf11bd59114","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ede1940704c1bf11bd59114: done: 0.010518 sec","job_status":"done","queue_duration_s":0.376097,"scheduling_latency_s":0.376111,"redis_calls":13,"redis_duration_s":0.0024749999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001661,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010518,"completed_at":"2026-07-18T06:33:03.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 138.054604] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66425229ea75c8bce96b99d1","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66425229ea75c8bce96b99d1: start","job_status":"start","queue_duration_s":0.387615,"scheduling_latency_s":0.387629}1402gitlab # [ 138.092044] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66425229ea75c8bce96b99d1","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66425229ea75c8bce96b99d1: done: 0.009446 sec","job_status":"done","queue_duration_s":0.387615,"scheduling_latency_s":0.387629,"redis_calls":13,"redis_duration_s":0.001718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009446,"completed_at":"2026-07-18T06:33:03.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 138.134959] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbf5671554ef629480004412","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbf5671554ef629480004412: start","job_status":"start","queue_duration_s":0.397922,"scheduling_latency_s":0.397935}1404gitlab # [ 138.135732] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbf5671554ef629480004412","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbf5671554ef629480004412: done: 0.010678 sec","job_status":"done","queue_duration_s":0.397922,"scheduling_latency_s":0.397935,"redis_calls":13,"redis_duration_s":0.001824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010678,"completed_at":"2026-07-18T06:33:03.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 138.143204] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2a1cf63b1a940b57275556d","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2a1cf63b1a940b57275556d: start","job_status":"start","queue_duration_s":0.409413,"scheduling_latency_s":0.409426}1406gitlab # [ 138.180600] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2a1cf63b1a940b57275556d","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2a1cf63b1a940b57275556d: done: 0.009838 sec","job_status":"done","queue_duration_s":0.409413,"scheduling_latency_s":0.409426,"redis_calls":13,"redis_duration_s":0.001624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009838,"completed_at":"2026-07-18T06:33:03.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 138.187664] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7602e7f63531e4dccd3e74b5","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7602e7f63531e4dccd3e74b5: start","job_status":"start","queue_duration_s":0.420048,"scheduling_latency_s":0.420061}1408gitlab # [ 138.198681] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7602e7f63531e4dccd3e74b5","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7602e7f63531e4dccd3e74b5: done: 0.009521 sec","job_status":"done","queue_duration_s":0.420048,"scheduling_latency_s":0.420061,"redis_calls":13,"redis_duration_s":0.0017530000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009521,"completed_at":"2026-07-18T06:33:03.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 138.241634] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9b5d4acb110d250c892b311","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9b5d4acb110d250c892b311: start","job_status":"start","queue_duration_s":0.43046,"scheduling_latency_s":0.430474}1410gitlab # [ 138.244610] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9b5d4acb110d250c892b311","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9b5d4acb110d250c892b311: done: 0.009535 sec","job_status":"done","queue_duration_s":0.43046,"scheduling_latency_s":0.430474,"redis_calls":13,"redis_duration_s":0.001733,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001243,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009535,"completed_at":"2026-07-18T06:33:03.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 138.291482] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2014c2296c9173bc82e24653","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2014c2296c9173bc82e24653: start","job_status":"start","queue_duration_s":0.441471,"scheduling_latency_s":0.441512}1412gitlab # [ 138.292244] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2014c2296c9173bc82e24653","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2014c2296c9173bc82e24653: done: 0.011393 sec","job_status":"done","queue_duration_s":0.441471,"scheduling_latency_s":0.441512,"redis_calls":13,"redis_duration_s":0.0015929999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011393,"completed_at":"2026-07-18T06:33:03.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 138.340700] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7c687c761d2e6dd610d5655","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7c687c761d2e6dd610d5655: start","job_status":"start","queue_duration_s":0.453649,"scheduling_latency_s":0.453663}1414gitlab # [ 138.341509] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7c687c761d2e6dd610d5655","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7c687c761d2e6dd610d5655: done: 0.009081 sec","job_status":"done","queue_duration_s":0.453649,"scheduling_latency_s":0.453663,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009081,"completed_at":"2026-07-18T06:33:03.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 138.400574] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b90971624c39076ddd3994a6","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b90971624c39076ddd3994a6: start","job_status":"start","queue_duration_s":0.463592,"scheduling_latency_s":0.463606}1416gitlab # [ 138.401519] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b90971624c39076ddd3994a6","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b90971624c39076ddd3994a6: done: 0.009049 sec","job_status":"done","queue_duration_s":0.463592,"scheduling_latency_s":0.463606,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009049,"completed_at":"2026-07-18T06:33:03.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 138.414431] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:03.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7f42a56ff4bc6c00bd40c57","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7f42a56ff4bc6c00bd40c57: start","job_status":"start","queue_duration_s":0.473522,"scheduling_latency_s":0.473536}1418gitlab # [ 138.467801] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7f42a56ff4bc6c00bd40c57","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7f42a56ff4bc6c00bd40c57: done: 1.551565 sec","job_status":"done","queue_duration_s":0.473522,"scheduling_latency_s":0.473536,"redis_calls":13,"redis_duration_s":0.001858,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":1.551565,"completed_at":"2026-07-18T06:33:05.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 138.502394] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54163e8d15e54c5adb8e57fe","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54163e8d15e54c5adb8e57fe: start","job_status":"start","queue_duration_s":2.060107,"scheduling_latency_s":2.060139}1420gitlab # [ 138.525638] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54163e8d15e54c5adb8e57fe","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54163e8d15e54c5adb8e57fe: done: 0.023304 sec","job_status":"done","queue_duration_s":2.060107,"scheduling_latency_s":2.060139,"redis_calls":13,"redis_duration_s":0.0018180000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023304,"completed_at":"2026-07-18T06:33:05.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 138.561031] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acbcb2c2c7f60b8f4485acb5","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acbcb2c2c7f60b8f4485acb5: start","job_status":"start","queue_duration_s":2.118666,"scheduling_latency_s":2.118703}1422gitlab # [ 138.591074] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acbcb2c2c7f60b8f4485acb5","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acbcb2c2c7f60b8f4485acb5: done: 0.029467 sec","job_status":"done","queue_duration_s":2.118666,"scheduling_latency_s":2.118703,"redis_calls":13,"redis_duration_s":0.002453,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001615,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029467,"completed_at":"2026-07-18T06:33:05.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 138.627119] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e21bd78cb1966dae3fba3fbe","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e21bd78cb1966dae3fba3fbe: start","job_status":"start","queue_duration_s":2.184463,"scheduling_latency_s":2.184514}1424gitlab # [ 138.650896] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e21bd78cb1966dae3fba3fbe","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e21bd78cb1966dae3fba3fbe: done: 0.023868 sec","job_status":"done","queue_duration_s":2.184463,"scheduling_latency_s":2.184514,"redis_calls":13,"redis_duration_s":0.003771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023868,"completed_at":"2026-07-18T06:33:05.627Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 138.687628] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68cd918b173c78b990f01151","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68cd918b173c78b990f01151: start","job_status":"start","queue_duration_s":2.245156,"scheduling_latency_s":2.245194}1426gitlab # [ 138.711052] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68cd918b173c78b990f01151","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68cd918b173c78b990f01151: done: 0.02333 sec","job_status":"done","queue_duration_s":2.245156,"scheduling_latency_s":2.245194,"redis_calls":13,"redis_duration_s":0.0036090000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001146,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002463,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02333,"completed_at":"2026-07-18T06:33:05.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 138.747570] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"496ec8e1f0faf5d8f4070829","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-496ec8e1f0faf5d8f4070829: start","job_status":"start","queue_duration_s":2.304609,"scheduling_latency_s":2.304647}1428gitlab # [ 138.778791] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"496ec8e1f0faf5d8f4070829","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-496ec8e1f0faf5d8f4070829: done: 0.031412 sec","job_status":"done","queue_duration_s":2.304609,"scheduling_latency_s":2.304647,"redis_calls":13,"redis_duration_s":0.0025369999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001718,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031412,"completed_at":"2026-07-18T06:33:05.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 138.814296] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b28790e34f64bc5fabe69f9","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b28790e34f64bc5fabe69f9: start","job_status":"start","queue_duration_s":2.37172,"scheduling_latency_s":2.371758}1430gitlab # [ 138.845267] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b28790e34f64bc5fabe69f9","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b28790e34f64bc5fabe69f9: done: 0.030836 sec","job_status":"done","queue_duration_s":2.37172,"scheduling_latency_s":2.371758,"redis_calls":13,"redis_duration_s":0.0024730000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001659,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030836,"completed_at":"2026-07-18T06:33:05.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 138.881445] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10e9ff1414f42eaff95dcfb8","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10e9ff1414f42eaff95dcfb8: start","job_status":"start","queue_duration_s":2.438957,"scheduling_latency_s":2.438988}1432gitlab # [ 138.914157] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10e9ff1414f42eaff95dcfb8","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10e9ff1414f42eaff95dcfb8: done: 0.032546 sec","job_status":"done","queue_duration_s":2.438957,"scheduling_latency_s":2.438988,"redis_calls":13,"redis_duration_s":0.002965,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032546,"completed_at":"2026-07-18T06:33:05.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 138.950375] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"affdb442d8671d81c365c890","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-affdb442d8671d81c365c890: start","job_status":"start","queue_duration_s":2.507761,"scheduling_latency_s":2.507803}1434gitlab # [ 138.981910] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"affdb442d8671d81c365c890","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-affdb442d8671d81c365c890: done: 0.031624 sec","job_status":"done","queue_duration_s":2.507761,"scheduling_latency_s":2.507803,"redis_calls":13,"redis_duration_s":0.002736,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031624,"completed_at":"2026-07-18T06:33:05.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 139.017846] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:05.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f320f356c9545aa76d57a2e5","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f320f356c9545aa76d57a2e5: start","job_status":"start","queue_duration_s":2.575308,"scheduling_latency_s":2.57535}1436gitlab # [ 139.042123] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f320f356c9545aa76d57a2e5","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f320f356c9545aa76d57a2e5: done: 0.02414 sec","job_status":"done","queue_duration_s":2.575308,"scheduling_latency_s":2.57535,"redis_calls":13,"redis_duration_s":0.0037410000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002507,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02414,"completed_at":"2026-07-18T06:33:06.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 139.044691] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6bd956e38412574063f4bab","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6bd956e38412574063f4bab: start","job_status":"start","queue_duration_s":2.602149,"scheduling_latency_s":2.602194}1438gitlab # [ 139.107881] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6bd956e38412574063f4bab","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6bd956e38412574063f4bab: done: 0.062944 sec","job_status":"done","queue_duration_s":2.602149,"scheduling_latency_s":2.602194,"redis_calls":13,"redis_duration_s":0.002586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001765,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.062944,"completed_at":"2026-07-18T06:33:06.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 139.145089] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"984fca03467375939b4e6aba","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-984fca03467375939b4e6aba: start","job_status":"start","queue_duration_s":2.702691,"scheduling_latency_s":2.702721}1440gitlab # [ 139.381265] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"984fca03467375939b4e6aba","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-984fca03467375939b4e6aba: done: 0.235864 sec","job_status":"done","queue_duration_s":2.702691,"scheduling_latency_s":2.702721,"redis_calls":13,"redis_duration_s":0.003509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002409,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.235864,"completed_at":"2026-07-18T06:33:06.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 139.440185] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e187a52f8056fcb990334155","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e187a52f8056fcb990334155: start","job_status":"start","queue_duration_s":2.997742,"scheduling_latency_s":2.997768}1442gitlab # [ 139.476562] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e187a52f8056fcb990334155","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e187a52f8056fcb990334155: done: 0.034039 sec","job_status":"done","queue_duration_s":2.997742,"scheduling_latency_s":2.997768,"redis_calls":13,"redis_duration_s":0.0037689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00257,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034039,"completed_at":"2026-07-18T06:33:06.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 139.513775] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ead93163ed7b91c525b56d2","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ead93163ed7b91c525b56d2: start","job_status":"start","queue_duration_s":3.071438,"scheduling_latency_s":3.071466}1444gitlab # [ 139.548467] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ead93163ed7b91c525b56d2","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ead93163ed7b91c525b56d2: done: 0.034412 sec","job_status":"done","queue_duration_s":3.071438,"scheduling_latency_s":3.071466,"redis_calls":13,"redis_duration_s":0.003719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002474,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034412,"completed_at":"2026-07-18T06:33:06.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 139.584367] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84b3645debf3f8e3a6e4e694","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84b3645debf3f8e3a6e4e694: start","job_status":"start","queue_duration_s":3.141752,"scheduling_latency_s":3.14179}1446gitlab # [ 139.614674] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84b3645debf3f8e3a6e4e694","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84b3645debf3f8e3a6e4e694: done: 0.0303 sec","job_status":"done","queue_duration_s":3.141752,"scheduling_latency_s":3.14179,"redis_calls":13,"redis_duration_s":0.0025239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0303,"completed_at":"2026-07-18T06:33:06.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 139.651054] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7b33ac0f04aedbdb03c604","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7b33ac0f04aedbdb03c604: start","job_status":"start","queue_duration_s":3.20846,"scheduling_latency_s":3.208513}1448gitlab # [ 139.673136] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7b33ac0f04aedbdb03c604","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7b33ac0f04aedbdb03c604: done: 0.02208 sec","job_status":"done","queue_duration_s":3.20846,"scheduling_latency_s":3.208513,"redis_calls":13,"redis_duration_s":0.003523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002423,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02208,"completed_at":"2026-07-18T06:33:06.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 139.709379] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ca521c1cd029353eafabdfe","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ca521c1cd029353eafabdfe: start","job_status":"start","queue_duration_s":3.266519,"scheduling_latency_s":3.266548}1450gitlab # [ 139.743962] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ca521c1cd029353eafabdfe","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ca521c1cd029353eafabdfe: done: 0.034132 sec","job_status":"done","queue_duration_s":3.266519,"scheduling_latency_s":3.266548,"redis_calls":13,"redis_duration_s":0.003638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002484,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034132,"completed_at":"2026-07-18T06:33:06.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 139.779996] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6d34f9d64e4900b04bef4ce","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6d34f9d64e4900b04bef4ce: start","job_status":"start","queue_duration_s":3.337378,"scheduling_latency_s":3.337415}1452gitlab # [ 139.810380] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6d34f9d64e4900b04bef4ce","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6d34f9d64e4900b04bef4ce: done: 0.030311 sec","job_status":"done","queue_duration_s":3.337378,"scheduling_latency_s":3.337415,"redis_calls":13,"redis_duration_s":0.002471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001655,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030311,"completed_at":"2026-07-18T06:33:06.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 139.846269] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc466aa02482e8e5cde265a9","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc466aa02482e8e5cde265a9: start","job_status":"start","queue_duration_s":3.403774,"scheduling_latency_s":3.403806}1454gitlab # [ 139.880601] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc466aa02482e8e5cde265a9","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc466aa02482e8e5cde265a9: done: 0.034201 sec","job_status":"done","queue_duration_s":3.403774,"scheduling_latency_s":3.403806,"redis_calls":13,"redis_duration_s":0.003675,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002482,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034201,"completed_at":"2026-07-18T06:33:06.857Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 139.883712] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ada340a31b069afe2add9ca","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ada340a31b069afe2add9ca: start","job_status":"start","queue_duration_s":3.441331,"scheduling_latency_s":3.441362}1456gitlab # [ 139.906743] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ada340a31b069afe2add9ca","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ada340a31b069afe2add9ca: done: 0.022876 sec","job_status":"done","queue_duration_s":3.441331,"scheduling_latency_s":3.441362,"redis_calls":13,"redis_duration_s":0.0036940000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002491,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022876,"completed_at":"2026-07-18T06:33:06.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 139.924077] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"658644066aab2abaf2f4ff80","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-658644066aab2abaf2f4ff80: start","job_status":"start","queue_duration_s":3.470653,"scheduling_latency_s":3.470685}1458gitlab # [ 139.939694] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"658644066aab2abaf2f4ff80","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-658644066aab2abaf2f4ff80: done: 0.026012 sec","job_status":"done","queue_duration_s":3.470653,"scheduling_latency_s":3.470685,"redis_calls":13,"redis_duration_s":0.009697,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026012,"completed_at":"2026-07-18T06:33:06.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 139.992875] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2d7f9515ded24dd1e19d744","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2d7f9515ded24dd1e19d744: start","job_status":"start","queue_duration_s":3.503522,"scheduling_latency_s":3.503548}1460gitlab # [ 139.993893] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2d7f9515ded24dd1e19d744","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2d7f9515ded24dd1e19d744: done: 0.0177 sec","job_status":"done","queue_duration_s":3.503522,"scheduling_latency_s":3.503548,"redis_calls":13,"redis_duration_s":0.0036550000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0177,"completed_at":"2026-07-18T06:33:06.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 140.047702] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0001292c76a795f7a0d46f59","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0001292c76a795f7a0d46f59: start","job_status":"start","queue_duration_s":3.522252,"scheduling_latency_s":3.522265}1462gitlab # [ 140.048591] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0001292c76a795f7a0d46f59","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0001292c76a795f7a0d46f59: done: 0.010195 sec","job_status":"done","queue_duration_s":3.522252,"scheduling_latency_s":3.522265,"redis_calls":13,"redis_duration_s":0.0026119999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010195,"completed_at":"2026-07-18T06:33:06.952Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 140.066736] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d6548423a0de473ad8bb25","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d6548423a0de473ad8bb25: start","job_status":"start","queue_duration_s":3.533513,"scheduling_latency_s":3.533526}1464gitlab # [ 140.078109] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d6548423a0de473ad8bb25","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d6548423a0de473ad8bb25: done: 0.009933 sec","job_status":"done","queue_duration_s":3.533513,"scheduling_latency_s":3.533526,"redis_calls":13,"redis_duration_s":0.002503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001727,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009933,"completed_at":"2026-07-18T06:33:06.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 140.119992] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab1bd1d9db887749707a6117","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab1bd1d9db887749707a6117: start","job_status":"start","queue_duration_s":3.544833,"scheduling_latency_s":3.544847}1466gitlab # [ 140.159544] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab1bd1d9db887749707a6117","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab1bd1d9db887749707a6117: done: 0.015985 sec","job_status":"done","queue_duration_s":3.544833,"scheduling_latency_s":3.544847,"redis_calls":13,"redis_duration_s":0.008206,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006583,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015985,"completed_at":"2026-07-18T06:33:06.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 140.160981] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"657a467b3eb7bbab6aec3a40","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-657a467b3eb7bbab6aec3a40: start","job_status":"start","queue_duration_s":3.565482,"scheduling_latency_s":3.565504}1468gitlab # [ 140.168326] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"657a467b3eb7bbab6aec3a40","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-657a467b3eb7bbab6aec3a40: done: 0.010303 sec","job_status":"done","queue_duration_s":3.565482,"scheduling_latency_s":3.565504,"redis_calls":13,"redis_duration_s":0.002465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010303,"completed_at":"2026-07-18T06:33:06.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 140.240502] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:06.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae8cee76b480373480fb96d4","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae8cee76b480373480fb96d4: start","job_status":"start","queue_duration_s":3.576827,"scheduling_latency_s":3.576841}1470gitlab # [ 140.241360] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:07.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae8cee76b480373480fb96d4","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae8cee76b480373480fb96d4: done: 0.00986 sec","job_status":"done","queue_duration_s":3.576827,"scheduling_latency_s":3.576841,"redis_calls":13,"redis_duration_s":0.002343,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00986,"completed_at":"2026-07-18T06:33:07.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 140.249822] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:07.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"639215aa25d369b3662587b7","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-639215aa25d369b3662587b7: start","job_status":"start","queue_duration_s":3.587737,"scheduling_latency_s":3.58775}1472gitlab # [ 140.261397] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:07.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"639215aa25d369b3662587b7","created_at":"2026-07-18T06:33:03.371Z","trace_propagation_headers":{"sentry-trace":"cd37b9f992a347dfb714b16277b56887-4b19ac43f71e41e6","baggage":"sentry-trace_id=cd37b9f992a347dfb714b16277b56887,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12f3b6bc0c42963fe29985fb7935185a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:03.419Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-639215aa25d369b3662587b7: done: 0.009698 sec","job_status":"done","queue_duration_s":3.587737,"scheduling_latency_s":3.58775,"redis_calls":13,"redis_duration_s":0.002426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009698,"completed_at":"2026-07-18T06:33:07.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 151.549201] setup-gitlab-start[2601]: W, [2026-07-18T06:33:18.526039 #2601] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1474gitlab # [ 173.279790] runuser[2600]: pam_unix(runuser:session): session closed for user gitlab1475gitlab # [ 173.283980] setup-gitlab-start[2808]: ++ jq -r .token1476gitlab # [ 173.286038] setup-gitlab-start[2809]: +++ date -d '+7 days' +%F1477gitlab # [ 173.291320] setup-gitlab-start[2807]: ++ curl -fs -X POST -H 'Authorization: Bearer cfbc5ced91d84ae4a8edb1c738f9822583f4eea4e2da432c09a18c7d991d1bb1' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-25"}' http://gitlab/api/v4/users/1/personal_access_tokens1478gitlab # [ 173.509797] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:40.486Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f52c7a5193695142d1ddddec","created_at":"2026-07-18T06:33:40.465Z","trace_propagation_headers":{"sentry-trace":"c2f1846027f14b40857cf9c84c7d8ab4-7669fad6226843e4","baggage":"sentry-trace_id=c2f1846027f14b40857cf9c84c7d8ab4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWENX0W90T94GZX7Z0ADH","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec43df877101bdaa602e26133163d28679fd667ab751704ab25424d50c00fbe2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:40.483Z","job_size_bytes":451,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f52c7a5193695142d1ddddec: start","job_status":"start","queue_duration_s":0.002894,"scheduling_latency_s":0.002942}1479gitlab # [ 173.556461] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:40 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 2471480gitlab # [ 173.561569] setup-gitlab-start[2553]: + TOKEN=glpat-ij5szn2Z6_E4gEWV0ZKbGm86MQp1OjEH.01.0w155ffyv1481gitlab # [ 173.561709] setup-gitlab-start[2553]: + test -n glpat-ij5szn2Z6_E4gEWV0ZKbGm86MQp1OjEH.01.0w155ffyv1482gitlab # [ 173.562895] setup-gitlab-start[2553]: + test glpat-ij5szn2Z6_E4gEWV0ZKbGm86MQp1OjEH.01.0w155ffyv '!=' null1483gitlab # [ 173.564129] setup-gitlab-start[2553]: + mkdir -p /var/lib/secrets1484gitlab # [ 173.572385] setup-gitlab-start[2553]: + echo glpat-ij5szn2Z6_E4gEWV0ZKbGm86MQp1OjEH.01.0w155ffyv1485gitlab # [ 173.574709] setup-gitlab-start[2553]: + chmod 644 /var/lib/secrets/gitlab-token1486gitlab # [ 173.579682] setup-gitlab-start[2553]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-ij5szn2Z6_E4gEWV0ZKbGm86MQp1OjEH.01.0w155ffyv' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1487gitlab # [ 173.722348] sidekiq-cluster[2051]: {"severity":"WARN","time":"2026-07-18T06:33:40.698Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f52c7a5193695142d1ddddec","created_at":"2026-07-18T06:33:40.465Z","trace_propagation_headers":{"sentry-trace":"c2f1846027f14b40857cf9c84c7d8ab4-7669fad6226843e4","baggage":"sentry-trace_id=c2f1846027f14b40857cf9c84c7d8ab4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWENX0W90T94GZX7Z0ADH","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec43df877101bdaa602e26133163d28679fd667ab751704ab25424d50c00fbe2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:40.483Z","job_size_bytes":451,"pid":2051,"sidekiq_tid":"lypf","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f52c7a5193695142d1ddddec: fail: 0.210912 sec","job_status":"fail","queue_duration_s":0.002894,"scheduling_latency_s":0.002942,"redis_calls":11,"redis_duration_s":0.003494,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000914,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001991,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000512076,"duration_s":0.210912,"completed_at":"2026-07-18T06:33:40.697Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.043314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 174.962760] gitaly[1355]: time="2026-07-18T06:33:41.940Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:51.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:41.939" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 174.963076] gitaly[1355]: time="2026-07-18T06:33:41.940Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:51.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:41.939" grpc.time_ms=0.529 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 175.095785] gitaly[1355]: time="2026-07-18T06:33:42.073Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.073" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1491gitlab # [ 175.096372] gitaly[1355]: time="2026-07-18T06:33:42.073Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.073" grpc.time_ms=0.336 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 175.137726] gitaly[1355]: time="2026-07-18T06:33:42.115Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC diskcache=1b071240-1bdf-4461-978a-30cc9e8f80c9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-18T06:33:52.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.082" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1493gitlab # [ 175.139693] gitaly[1355]: time="2026-07-18T06:33:42.115Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=322 command.oublock=32 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-18T06:33:52.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.082" grpc.time_ms=32.449 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1494gitlab # [ 175.151553] gitaly[1355]: time="2026-07-18T06:33:42.129Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.058" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.128" pid=1355 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1495gitlab # [ 175.153096] gitaly[1355]: time="2026-07-18T06:33:42.129Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.058" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.128" grpc.time_ms=0.352 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1496gitlab # [ 175.165983] gitaly[1355]: time="2026-07-18T06:33:42.143Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:52.062" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.132" grpc.time_ms=10.52 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1497gitlab # [ 175.210913] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.187Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"fc745b343fa5e240fd7c3d65","created_at":"2026-07-18T06:33:42.154Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.184Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-fc745b343fa5e240fd7c3d65: start","job_status":"start","queue_duration_s":0.002768,"scheduling_latency_s":0.002823}1498gitlab # [ 175.232889] gitaly[1355]: time="2026-07-18T06:33:42.210Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.140" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.210" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1499gitlab # [ 175.243267] gitaly[1355]: time="2026-07-18T06:33:42.210Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.140" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.210" grpc.time_ms=0.198 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1500gitlab # [ 175.252578] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.229Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"fc745b343fa5e240fd7c3d65","created_at":"2026-07-18T06:33:42.154Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.184Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-fc745b343fa5e240fd7c3d65: done: 0.042275 sec","job_status":"done","queue_duration_s":0.002768,"scheduling_latency_s":0.002823,"redis_calls":10,"redis_duration_s":0.001536,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.028696106,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.042275,"completed_at":"2026-07-18T06:33:42.229Z","load_balancing_strategy":"primary","db_duration_s":0.009072,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1501gitlab # [ 175.254171] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.230Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"076069776859563d5f62cd1f","created_at":"2026-07-18T06:33:42.196Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.204Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-076069776859563d5f62cd1f: start","job_status":"start","queue_duration_s":0.026484,"scheduling_latency_s":0.026498}1502gitlab # [ 175.332210] gitaly[1355]: time="2026-07-18T06:33:42.309Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC diskcache=5ff85736-4ac5-4db6-81f1-314aa400755d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-18T06:33:52.143" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.214" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1503gitlab # [ 175.332509] gitaly[1355]: time="2026-07-18T06:33:42.310Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=55 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=32 command.real_time_ms=68 command.spawn_token_fork_ms=0 command.system_time_ms=47 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-18T06:33:52.143" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.214" grpc.time_ms=95.397 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1504gitlab # [ 175.340509] gitaly[1355]: time="2026-07-18T06:33:42.318Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.246" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.317" pid=1355 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1505gitlab # [ 175.341404] gitaly[1355]: time="2026-07-18T06:33:42.318Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.246" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.317" grpc.time_ms=0.751 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1506gitlab # [ 175.343875] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.321Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"076069776859563d5f62cd1f","created_at":"2026-07-18T06:33:42.196Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.204Z","job_size_bytes":2,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-076069776859563d5f62cd1f: done: 0.090251 sec","job_status":"done","queue_duration_s":0.026484,"scheduling_latency_s":0.026498,"redis_calls":14,"redis_duration_s":0.002713,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.049003993,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.090251,"completed_at":"2026-07-18T06:33:42.321Z","load_balancing_strategy":"primary","db_duration_s":0.020222,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1507gitlab # [ 175.395895] gitaly[1355]: time="2026-07-18T06:33:42.333Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:52.253" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.323" grpc.time_ms=9.444 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1508gitlab # [ 175.396691] gitaly[1355]: time="2026-07-18T06:33:42.356Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:52.285" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.356" grpc.time_ms=0.204 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1509gitlab # [ 175.397609] gitaly[1355]: time="2026-07-18T06:33:42.365Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-18T06:33:52.294" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.364" grpc.time_ms=0.363 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1510gitlab # [ 175.656485] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.633Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1834722e133ccbe721ca2f39","created_at":"2026-07-18T06:33:42.626Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.631Z","job_size_bytes":5,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1834722e133ccbe721ca2f39: start","job_status":"start","queue_duration_s":0.001853,"scheduling_latency_s":0.001898}1511gitlab # [ 175.708572] gitaly[1355]: time="2026-07-18T06:33:42.686Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWEZ02KBBZB6VD6EASXBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:52.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:42.679" grpc.time_ms=6 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1512gitlab # [ 175.761415] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.738Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1834722e133ccbe721ca2f39","created_at":"2026-07-18T06:33:42.626Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.631Z","job_size_bytes":5,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1834722e133ccbe721ca2f39: done: 0.10545 sec","job_status":"done","queue_duration_s":0.001853,"scheduling_latency_s":0.001898,"redis_calls":12,"redis_duration_s":0.003512,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000897,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001898,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289422,"duration_s":0.10545,"completed_at":"2026-07-18T06:33:42.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017673,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1513gitlab # [ 175.805583] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.740Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c68a42e6d9b595567bfa9763","created_at":"2026-07-18T06:33:42.651Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.655Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c68a42e6d9b595567bfa9763: start","job_status":"start","queue_duration_s":0.084688,"scheduling_latency_s":0.084702}1514gitlab # [ 175.946059] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.923Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c68a42e6d9b595567bfa9763","created_at":"2026-07-18T06:33:42.651Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.655Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c68a42e6d9b595567bfa9763: done: 0.183357 sec","job_status":"done","queue_duration_s":0.084688,"scheduling_latency_s":0.084702,"redis_calls":11,"redis_duration_s":0.002504,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000675,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.16,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.16,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002816,"duration_s":0.183357,"completed_at":"2026-07-18T06:33:42.923Z","load_balancing_strategy":"primary","db_duration_s":0.098222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 176.022292] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:42.999Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f727484ab9899c61058ee964","created_at":"2026-07-18T06:33:42.660Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.668Z","job_size_bytes":89,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f727484ab9899c61058ee964: start","job_status":"start","queue_duration_s":0.331229,"scheduling_latency_s":0.331278}1516gitlab # [ 176.066178] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:43 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 24671517gitlab # [ 176.069929] setup-gitlab-start[2553]: + mkdir -p /tmp/test-flake1518gitlab # [ 176.072672] setup-gitlab-start[2553]: + cd /tmp/test-flake1519gitlab # [ 176.073883] setup-gitlab-start[2553]: + git init -b master1520gitlab # [ 176.101329] setup-gitlab-start[2856]: Initialized empty Git repository in /tmp/test-flake/.git/1521gitlab # [ 176.101952] setup-gitlab-start[2553]: + git config user.name test1522gitlab # [ 176.104492] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:43.081Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f727484ab9899c61058ee964","created_at":"2026-07-18T06:33:42.660Z","trace_propagation_headers":{"sentry-trace":"5a3fa1328760426f95d78e79587e36c4-3fda918ab01c4e96","baggage":"sentry-trace_id=5a3fa1328760426f95d78e79587e36c4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWEZ02KBBZB6VD6EASXBC","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:42.668Z","job_size_bytes":89,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f727484ab9899c61058ee964: done: 0.082353 sec","job_status":"done","queue_duration_s":0.331229,"scheduling_latency_s":0.331278,"redis_calls":11,"redis_duration_s":0.0035050000000000003,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001915,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000430222,"duration_s":0.082353,"completed_at":"2026-07-18T06:33:43.081Z","load_balancing_strategy":"primary","db_duration_s":0.025297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 176.157591] setup-gitlab-start[2553]: + git config user.email test@example.com1524gitlab # [ 176.160230] setup-gitlab-start[2553]: + cat1525gitlab # [ 176.162397] setup-gitlab-start[2553]: + git add flake.nix1526gitlab # [ 176.167791] setup-gitlab-start[2553]: + git commit -m 'initial commit'1527gitlab # [ 176.178447] setup-gitlab-start[2861]: [master (root-commit) 87195bc] initial commit1528gitlab # [ 176.179567] setup-gitlab-start[2861]: 1 file changed, 10 insertions(+)1529gitlab # [ 176.180505] setup-gitlab-start[2861]: create mode 100644 flake.nix1530gitlab # [ 176.181388] setup-gitlab-start[2553]: + git remote add origin http://oauth2:glpat-ij5szn2Z6_E4gEWV0ZKbGm86MQp1OjEH.01.0w155ffyv@gitlab/root/test-flake.git1531gitlab # [ 176.187455] setup-gitlab-start[2553]: + git push -u origin master1532gitlab # [ 176.309622] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:43 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 961533gitlab # [ 176.389316] gitaly[1355]: time="2026-07-18T06:33:43.366Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHKQN39ASGMNZ9WMFSJK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:43.358" grpc.time_ms=7.099 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1534gitlab # [ 176.390125] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:43 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 781535gitlab # [ 176.641572] gitaly[1355]: time="2026-07-18T06:33:43.618Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:53.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:43.608" grpc.time_ms=10.106 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1536gitlab # [ 176.676141] gitaly[1355]: time="2026-07-18T06:33:43.653Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-18T06:34:13.047" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-18T06:33:43.647" grpc.time_ms=5.906 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 176.707292] gitaly[1355]: time="2026-07-18T06:33:43.683Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 duration_ms=124ns method=POST pid=1355 status=200 url="http://unix/api/v4/internal/allowed"1538gitlab # [ 176.730149] gitaly[1355]: time="2026-07-18T06:33:43.707Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 duration_ms=23ns method=POST pid=1355 status=200 url="http://unix/api/v4/internal/pre_receive"1539gitlab # [ 176.731295] gitaly[1355]: time="2026-07-18T06:33:43.708Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:43.558" grpc.time_ms=150.788 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1540gitlab # [ 176.776472] gitaly[1355]: time="2026-07-18T06:33:43.754Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:43.752" grpc.time_ms=1.014 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1541gitlab # [ 176.822392] gitaly[1355]: time="2026-07-18T06:33:43.800Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:43.799" grpc.time_ms=0.589 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1542gitlab # [ 176.858912] gitaly[1355]: time="2026-07-18T06:33:43.836Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:43.835" grpc.time_ms=0.865 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1543gitlab # [ 176.926413] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:43.903Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 87195bc874ea0a4ada07b3774f27b294e19614b7 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"410c409140b829cda1bdfca4","created_at":"2026-07-18T06:33:43.896Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-44477f36b0f04ba1","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:43.901Z","job_size_bytes":185,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-410c409140b829cda1bdfca4: start","job_status":"start","queue_duration_s":0.001303,"scheduling_latency_s":0.001334}1544gitlab # [ 176.956086] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:43.933Z","message":"changes: \"0000000000000000000000000000000000000000 87195bc874ea0a4ada07b3774f27b294e19614b7 refs/heads/master\\n\""}1545gitlab # [ 176.979333] gitaly[1355]: time="2026-07-18T06:33:43.956Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:53.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:43.948" grpc.time_ms=7.91 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1546gitlab # [ 176.993084] gitaly[1355]: time="2026-07-18T06:33:43.965Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:53.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:43.957" grpc.time_ms=7.76 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 177.006726] gitaly[1355]: time="2026-07-18T06:33:43.984Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:53.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:43.976" grpc.time_ms=7.861 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1548gitlab # [ 177.019638] gitaly[1355]: time="2026-07-18T06:33:43.988Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1566 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=7 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-18T06:33:53.021" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:43.971" grpc.time_ms=16.461 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1549gitlab # [ 177.033220] gitaly[1355]: time="2026-07-18T06:33:43.995Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 duration_ms=120ns method=POST pid=1355 status=200 url="http://unix/api/v4/internal/post_receive"1550gitlab # [ 177.033528] gitaly[1355]: time="2026-07-18T06:33:43.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:43.874" grpc.time_ms=121.784 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1551gitlab # [ 177.052389] gitaly[1355]: time="2026-07-18T06:33:44.030Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-18T06:33:54.002" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:44.022" grpc.time_ms=7.634 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1552gitlab # [ 177.059326] gitaly[1355]: time="2026-07-18T06:33:44.036Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 diskcache=36c479eb-fb73-4176-988d-f3df10baa61d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:43.497" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1553gitlab # [ 177.059993] gitaly[1355]: time="2026-07-18T06:33:44.036Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=199 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13162 command.oublock=32 command.real_time_ms=525 command.spawn_token_fork_ms=0 command.system_time_ms=132 command.user_time_ms=67 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:43.497" grpc.time_ms=538.661 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1554gitlab # [ 177.090528] gitlab-workhorse[1290]: time="2026-07-18T06:33:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521555gitlab # [ 177.090843] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:44 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6781556gitlab # [ 177.094663] setup-gitlab-start[2865]: To http://gitlab/root/test-flake.git1557gitlab # [ 177.094898] setup-gitlab-start[2865]: * [new branch] master -> master1558gitlab # [ 177.099222] setup-gitlab-start[2865]: branch 'master' set up to track 'origin/master'.1559gitlab # [ 177.100885] systemd[1]: Finished setup-gitlab.service.1560gitlab # [ 177.104294] systemd[1]: Starting nixbot CI service...1561gitlab # [ 177.112817] gitaly[1355]: time="2026-07-18T06:33:44.090Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3941 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:44.090" grpc.time_ms=0.314 pid=1355 span.kind=server system=grpc1562gitlab # [ 177.139755] gitaly[1355]: time="2026-07-18T06:33:44.117Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3943 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:44.116" grpc.time_ms=0.421 pid=1355 span.kind=server system=grpc1563gitlab # [ 177.158069] gitaly[1355]: time="2026-07-18T06:33:44.134Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 diskcache=a88de823-a8b5-4b67-b092-a7d4caef19a9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-18T06:33:54.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:44.068" pid=1355 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1564gitlab # [ 177.158260] gitaly[1355]: time="2026-07-18T06:33:44.135Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=42 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4834 command.oublock=8 command.real_time_ms=55 command.spawn_token_fork_ms=0 command.system_time_ms=25 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-18T06:33:54.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:44.068" grpc.time_ms=66.683 pid=1355 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1565gitlab # [ 177.167771] gitaly[1355]: time="2026-07-18T06:33:44.145Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-18T06:33:54.125" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:44.145" grpc.time_ms=0.107 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1566gitlab # [ 177.178739] gitaly[1355]: time="2026-07-18T06:33:44.156Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-18T06:33:54.128" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:44.149" grpc.time_ms=7.147 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1567gitlab # [ 177.234831] gitaly[1355]: time="2026-07-18T06:33:44.212Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-18T06:34:14.006" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.206" grpc.time_ms=5.885 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1568gitlab # [ 177.251707] gitaly[1355]: time="2026-07-18T06:33:44.229Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-18T06:33:54.208" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:44.228" grpc.time_ms=0.184 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 177.256882] gitaly[1355]: time="2026-07-18T06:33:44.234Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-18T06:33:54.211" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-18T06:33:44.231" grpc.time_ms=3.298 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1570gitlab # [ 177.270968] gitaly[1355]: time="2026-07-18T06:33:44.248Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:54.223" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:44.243" grpc.time_ms=4.87 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1571gitlab # [ 177.278344] gitaly[1355]: time="2026-07-18T06:33:44.255Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:54.231" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:44.251" grpc.time_ms=4.713 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 177.293317] gitaly[1355]: time="2026-07-18T06:33:44.270Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-18T06:34:14.062" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.262" grpc.time_ms=8.363 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab: (finished: waiting for unit setup-gitlab.service, in 178.20 seconds)1574gitlab # [ 177.527943] gitaly[1355]: time="2026-07-18T06:33:44.504Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-18T06:33:54.002" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.503" grpc.time_ms=1.427 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab: waiting for unit nixbot.service1576gitlab # [ 177.553947] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.531Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 87195bc874ea0a4ada07b3774f27b294e19614b7 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"410c409140b829cda1bdfca4","created_at":"2026-07-18T06:33:43.896Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-44477f36b0f04ba1","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:43.901Z","job_size_bytes":185,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-410c409140b829cda1bdfca4: done: 0.627953 sec","job_status":"done","queue_duration_s":0.001303,"scheduling_latency_s":0.001334,"gitaly_calls":13,"gitaly_duration_s":0.189114,"redis_calls":200,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.038895,"redis_read_bytes":941,"redis_write_bytes":19810,"redis_cache_calls":14,"redis_cache_duration_s":0.002503,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":32,"redis_feature_flag_duration_s":0.006528,"redis_feature_flag_read_bytes":540,"redis_feature_flag_write_bytes":4279,"redis_queues_calls":8,"redis_queues_duration_s":0.000891,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004608,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.014215,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01015,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":51,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":51,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":51,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.161,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.179,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.326136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000402286,"exclusive_lock_hold_duration_s":0.022215951,"duration_s":0.627953,"completed_at":"2026-07-18T06:33:44.530Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075106,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1577gitlab # [ 177.604150] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.579Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"689a6f6a94dc5a66f92c951d","created_at":"2026-07-18T06:33:43.989Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:43.996Z","job_size_bytes":46,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-689a6f6a94dc5a66f92c951d: start","job_status":"start","queue_duration_s":0.582937,"scheduling_latency_s":0.582981}1578gitlab # [ 177.674347] gitaly[1355]: time="2026-07-18T06:33:44.651Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-18T12:33:44.646" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:44.646" grpc.time_ms=4.576 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1579gitlab # [ 177.706258] gitaly[1355]: time="2026-07-18T06:33:44.683Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-18T06:34:14.474" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.675" grpc.time_ms=8.282 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 177.732606] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.709Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"689a6f6a94dc5a66f92c951d","created_at":"2026-07-18T06:33:43.989Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:43.996Z","job_size_bytes":46,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-689a6f6a94dc5a66f92c951d: done: 0.130896 sec","job_status":"done","queue_duration_s":0.582937,"scheduling_latency_s":0.582981,"gitaly_calls":2,"gitaly_duration_s":0.032676,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007348,"redis_read_bytes":68,"redis_write_bytes":4889,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001392,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000709,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003161,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000315404,"duration_s":0.130896,"completed_at":"2026-07-18T06:33:44.709Z","load_balancing_strategy":"primary","db_duration_s":0.012031,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1581gitlab # [ 177.782459] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.759Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"42b94e37c18021c8d3a8de07","created_at":"2026-07-18T06:33:44.017Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.019Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-42b94e37c18021c8d3a8de07: start","job_status":"start","queue_duration_s":0.739557,"scheduling_latency_s":0.739598}1582gitlab # [ 177.838119] gitaly[1355]: time="2026-07-18T06:33:44.815Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 diskcache=833c6f1b-9587-4bcb-b05f-9d879c7efbad grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-18T12:33:44.792" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.792" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 177.848761] gitaly[1355]: time="2026-07-18T06:33:44.815Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1286 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-18T12:33:44.792" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.792" grpc.time_ms=23.181 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 177.869125] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.846Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"42b94e37c18021c8d3a8de07","created_at":"2026-07-18T06:33:44.017Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.019Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-42b94e37c18021c8d3a8de07: done: 0.08708 sec","job_status":"done","queue_duration_s":0.739557,"scheduling_latency_s":0.739598,"gitaly_calls":1,"gitaly_duration_s":0.025803,"redis_calls":15,"redis_duration_s":0.004457,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000605,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001577,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000941,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001334,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058785125,"duration_s":0.08708,"completed_at":"2026-07-18T06:33:44.846Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1585gitlab # [ 177.916272] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.893Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e838c77cd70e3f11e00a88a7","created_at":"2026-07-18T06:33:44.142Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.143Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e838c77cd70e3f11e00a88a7: start","job_status":"start","queue_duration_s":0.749697,"scheduling_latency_s":0.749748}1586gitlab # [ 177.957084] gitaly[1355]: time="2026-07-18T06:33:44.934Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-18T06:34:14.727" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:44.928" grpc.time_ms=6.338 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1587gitlab # [ 177.977654] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.954Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e838c77cd70e3f11e00a88a7","created_at":"2026-07-18T06:33:44.142Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.143Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e838c77cd70e3f11e00a88a7: done: 0.061249 sec","job_status":"done","queue_duration_s":0.749697,"scheduling_latency_s":0.749748,"gitaly_calls":1,"gitaly_duration_s":0.020514,"redis_calls":16,"redis_duration_s":0.003691,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000286,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001689,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000628,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061249,"completed_at":"2026-07-18T06:33:44.954Z","load_balancing_strategy":"primary","db_duration_s":0.002495,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1588gitlab # [ 177.978968] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:44.955Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"dd748477d992fa5d66cead4c","created_at":"2026-07-18T06:33:44.465Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8af98d9a2c4616877e5ab54aa2d02281160d2b0febb384ec40cdcbdcc801a0a2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.471Z","job_size_bytes":467,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-dd748477d992fa5d66cead4c: start","job_status":"start","queue_duration_s":0.483651,"scheduling_latency_s":0.483672}1589gitlab # [ 178.092258] gitaly[1355]: time="2026-07-18T06:33:45.069Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:15.067" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:45.068" grpc.time_ms=0.832 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 178.190663] nixbot[2971]: {"ts": "2026-07-18T06:33:45.167Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1591gitlab # [ 178.222503] nixbot[2971]: {"ts": "2026-07-18T06:33:45.199Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1592gitlab # [ 178.238822] gitaly[1355]: time="2026-07-18T06:33:45.216Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-18T06:33:55.188" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:45.208" grpc.time_ms=7.298 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 178.267646] nixbot[2971]: {"ts": "2026-07-18T06:33:45.224Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1594gitlab # [ 178.267757] nixbot[2971]: {"ts": "2026-07-18T06:33:45.230Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1595gitlab # [ 178.268893] nixbot[2971]: {"ts": "2026-07-18T06:33:45.234Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1596gitlab # [ 178.270091] nixbot[2971]: {"ts": "2026-07-18T06:33:45.239Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1597gitlab # [ 178.270826] nixbot[2971]: {"ts": "2026-07-18T06:33:45.242Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1598gitlab # [ 178.271878] nixbot[2971]: {"ts": "2026-07-18T06:33:45.244Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1599gitlab # [ 178.294535] nixbot[2971]: {"ts": "2026-07-18T06:33:45.272Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1600gitlab # [ 178.296638] nixbot[2971]: {"ts": "2026-07-18T06:33:45.274Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1601gitlab # [ 178.300887] nixbot[2971]: {"ts": "2026-07-18T06:33:45.278Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1602gitlab # [ 178.305362] nixbot[2971]: {"ts": "2026-07-18T06:33:45.283Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1603gitlab # [ 178.309740] nixbot[2971]: {"ts": "2026-07-18T06:33:45.287Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1604gitlab # [ 178.314138] nixbot[2971]: {"ts": "2026-07-18T06:33:45.291Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1605gitlab # [ 178.320460] nixbot[2971]: {"ts": "2026-07-18T06:33:45.298Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1606gitlab # [ 178.322713] nixbot[2971]: {"ts": "2026-07-18T06:33:45.300Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1607gitlab # [ 178.324271] nixbot[2971]: {"ts": "2026-07-18T06:33:45.301Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1608gitlab # [ 178.326058] nixbot[2971]: {"ts": "2026-07-18T06:33:45.303Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1609gitlab # [ 178.328866] nixbot[2971]: {"ts": "2026-07-18T06:33:45.306Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1610gitlab # [ 178.330770] nixbot[2971]: {"ts": "2026-07-18T06:33:45.308Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1611gitlab # [ 178.332501] nixbot[2971]: {"ts": "2026-07-18T06:33:45.310Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1612gitlab # [ 178.335973] nixbot[2971]: {"ts": "2026-07-18T06:33:45.313Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1613gitlab # [ 178.351961] gitaly[1355]: time="2026-07-18T06:33:45.329Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:15.328" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:45.328" grpc.time_ms=0.687 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1614gitlab # [ 178.367959] gitaly[1355]: time="2026-07-18T06:33:45.345Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=51 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-18T06:34:15.343" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:45.343" grpc.time_ms=1.6 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 178.479316] gitaly[1355]: time="2026-07-18T06:33:45.456Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:55.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:45.450" grpc.time_ms=6.306 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1616gitlab # [ 178.635589] nixbot[2971]: INFO: Started server process [2971]1617gitlab # [ 178.638119] nixbot[2971]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1618gitlab # [ 178.653167] nixbot[2971]: INFO: ::ffff:127.0.0.1:40354 - "GET /health HTTP/1.1" 200 OK1619gitlab # [ 178.657377] systemd[1]: Started nixbot CI service.1620gitlab # [ 178.658681] systemd[1]: Reached target Multi-User System.1621gitlab # [ 178.660492] systemd[1]: Startup finished in 1.124s (kernel) + 4.011s (initrd) + 2min 53.524s (userspace) = 2min 58.660s.1622gitlab # [ 178.930340] gitaly[1355]: time="2026-07-18T06:33:45.907Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=971 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-18T06:34:15.891" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:45.892" grpc.time_ms=15.199 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1623gitlab # [ 178.964683] gitaly[1355]: time="2026-07-18T06:33:45.942Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=957 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWKWTDVCKJDS34GX82PWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-18T06:34:15.298" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:45.899" grpc.time_ms=42.807 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1624gitlab # [ 178.993096] gitaly[1355]: time="2026-07-18T06:33:45.970Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWHPX5Y2QHW5XD0VZ6F53 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-18T06:34:15.740" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:45.941" grpc.time_ms=28.849 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1625gitlab # [ 179.029219] sidekiq-cluster[2051]: {"severity":"WARN","time":"2026-07-18T06:33:46.006Z","class":"CreatePipelineWorker","correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"87195bc874ea0a4ada07b3774f27b294e19614b7","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"87195bc874ea0a4ada07b3774f27b294e19614b7","ref":"refs/heads/master"},"retry":0}1626gitlab # [ 179.033372] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:46.010Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"dd748477d992fa5d66cead4c","created_at":"2026-07-18T06:33:44.465Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-7bcb7213921a4789","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8af98d9a2c4616877e5ab54aa2d02281160d2b0febb384ec40cdcbdcc801a0a2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.471Z","job_size_bytes":467,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-dd748477d992fa5d66cead4c: done: 1.055268 sec","job_status":"done","queue_duration_s":0.483651,"scheduling_latency_s":0.483672,"gitaly_calls":7,"gitaly_duration_s":0.298326,"redis_calls":53,"redis_duration_s":0.013024,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.000968,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006713,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003576,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.59717,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000224609,"duration_s":1.055268,"completed_at":"2026-07-18T06:33:46.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084493,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1627gitlab # [ 179.039748] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:46.017Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2a759ab6899c7b9ac6d29237","created_at":"2026-07-18T06:33:44.694Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-a9127673677a433b","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.699Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2a759ab6899c7b9ac6d29237: start","job_status":"start","queue_duration_s":1.317555,"scheduling_latency_s":1.317572}1628gitlab # [ 179.070926] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:46.048Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2a759ab6899c7b9ac6d29237","created_at":"2026-07-18T06:33:44.694Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-a9127673677a433b","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:44.699Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2a759ab6899c7b9ac6d29237: done: 0.031303 sec","job_status":"done","queue_duration_s":1.317555,"scheduling_latency_s":1.317572,"redis_calls":19,"redis_duration_s":0.004826,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000984,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000474,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002791,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004535213,"duration_s":0.031303,"completed_at":"2026-07-18T06:33:46.048Z","load_balancing_strategy":"primary","db_duration_s":0.006077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1629gitlab # [ 179.113055] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:46.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"a5808a98a60b069efebafe93","created_at":"2026-07-18T06:33:46.040Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-89f909ebb2c54b4e","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:46.043Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-a5808a98a60b069efebafe93: start","job_status":"start","queue_duration_s":0.009504,"scheduling_latency_s":0.009518}1630gitlab # [ 179.125139] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:46.102Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"a5808a98a60b069efebafe93","created_at":"2026-07-18T06:33:46.040Z","trace_propagation_headers":{"sentry-trace":"6c20f69fcd0d4eb68dd7bb3d00bccf35-89f909ebb2c54b4e","baggage":"sentry-trace_id=6c20f69fcd0d4eb68dd7bb3d00bccf35,sentry-release=Unknown"},"correlation_id":"01KXSYWHPX5Y2QHW5XD0VZ6F53","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:46.043Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-a5808a98a60b069efebafe93: done: 0.049469 sec","job_status":"done","queue_duration_s":0.009504,"scheduling_latency_s":0.009518,"redis_calls":11,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000192203,"duration_s":0.049469,"completed_at":"2026-07-18T06:33:46.102Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1631gitlab # [ 179.308345] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 6581632gitlab # [ 179.310285] nixbot[2971]: {"ts": "2026-07-18T06:33:46.287Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1633gitlab # [ 179.324777] nixbot[2971]: {"ts": "2026-07-18T06:33:46.302Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1634gitlab # [ 179.404815] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 711635gitlab # [ 179.406166] nixbot[2971]: {"ts": "2026-07-18T06:33:46.383Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1636gitlab # [ 179.408243] nixbot[2971]: {"ts": "2026-07-18T06:33:46.385Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1637gitlab # [ 179.540768] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1221638gitlab # [ 179.542201] nixbot[2971]: {"ts": "2026-07-18T06:33:46.519Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1639gitlab # [ 179.651471] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1001640gitlab # [ 179.656962] nixbot[2971]: {"ts": "2026-07-18T06:33:46.630Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1641gitlab: (finished: waiting for unit nixbot.service, in 2.13 seconds)1642gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1643gitlab # [ 179.696130] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 311644gitlab # [ 179.701180] nixbot[2971]: {"ts": "2026-07-18T06:33:46.676Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1645gitlab # [ 179.717153] nixbot[2971]: INFO: ::ffff:127.0.0.1:40356 - "GET /health HTTP/1.1" 200 OK1646gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1647subtest: project discovered and webhook registered1648gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1649gitlab # [ 179.748147] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 441650gitlab # [ 179.753149] nixbot[2971]: {"ts": "2026-07-18T06:33:46.726Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1651gitlab # [ 179.799530] gitaly[1355]: time="2026-07-18T06:33:46.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWMZG9ZYR30X4WYV5G4TS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-18T06:34:16.066" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:46.767" grpc.time_ms=9.399 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652gitlab # [ 179.832750] gitaly[1355]: time="2026-07-18T06:33:46.784Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWMZG9ZYR30X4WYV5G4TS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:16.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:46.783" grpc.time_ms=0.417 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds)1654(finished: subtest: project discovered and webhook registered, in 0.13 seconds)1655subtest: push delivers the webhook1656gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1657gitlab # [ 179.852213] gitaly[1355]: time="2026-07-18T06:33:46.797Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWMZG9ZYR30X4WYV5G4TS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-18T06:33:56.119" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:46.790" grpc.time_ms=7.112 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1658gitlab # [ 179.864606] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 851659gitlab # [ 179.866761] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1011660gitlab # [ 179.869101] nixbot[2971]: {"ts": "2026-07-18T06:33:46.839Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1661gitlab # [ 179.910852] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 201662gitlab # [ 179.915083] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 501663gitlab # [ 179.919466] nixbot[2971]: {"ts": "2026-07-18T06:33:46.892Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1664gitlab # [ 179.923908] nixbot[2971]: {"ts": "2026-07-18T06:33:46.894Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1665gitlab # [ 179.972164] gitaly[1355]: time="2026-07-18T06:33:46.949Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN49FTVNTS0CR5RQSRPE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:46.939" grpc.time_ms=10.032 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1666gitlab # [ 179.985680] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 611667gitlab # Enumerating objects: 5, done.1668gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1669gitlab # Delta compression using up to 4 threads1670gitlab # [ 179.991092] gitaly[1355]: time="2026-07-18T06:33:46.966Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN58PKSJDR55EYFJ5WVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:46.956" grpc.time_ms=10.33 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1671gitlab # Compressing objects: 50% (1/2)[ 180.006406] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:46 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 461672gitlab # Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1673gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done.1674gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1675gitlab # [ 180.054088] gitaly[1355]: time="2026-07-18T06:33:47.031Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN6RZDZS5NRGXRFWT4TC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.022" pid=1355 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1676gitlab # [ 180.054480] gitaly[1355]: time="2026-07-18T06:33:47.031Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN6RZDZS5NRGXRFWT4TC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.022" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1677gitlab # [ 180.055146] gitaly[1355]: time="2026-07-18T06:33:47.031Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN6RZDZS5NRGXRFWT4TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.022" grpc.time_ms=8.545 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1678gitlab # [ 180.055775] gitlab-workhorse[1290]: time="2026-07-18T06:33:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491679gitlab # [ 180.056507] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 641680gitlab # [ 180.103875] gitaly[1355]: time="2026-07-18T06:33:47.081Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN8W0PYM1RJT2A464V13 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.066" pid=1355 remote_ip="2001:db8:1::1" request_sha=234e46c74208ec1c83f43c4966bd78960b198ddb response_bytes=229 span.kind=server system=grpc1681gitlab # [ 180.104359] gitaly[1355]: time="2026-07-18T06:33:47.081Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN8W0PYM1RJT2A464V13 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.066" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1682gitlab # [ 180.106633] gitaly[1355]: time="2026-07-18T06:33:47.082Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN8W0PYM1RJT2A464V13 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.066" grpc.time_ms=16.412 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1683gitlab # [ 180.107594] gitlab-workhorse[1290]: time="2026-07-18T06:33:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2291684gitlab # [ 180.108074] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 229 "" "git/2.54.0" 471685gitlab # [ 180.233989] gitaly[1355]: time="2026-07-18T06:33:47.211Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=7 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNCQ911DXGKPXJMNGN5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.198" grpc.time_ms=13.538 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1686gitlab # [ 180.235824] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 531687gitlab # [ 180.250503] gitaly[1355]: time="2026-07-18T06:33:47.228Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-18T06:33:57.151" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.222" grpc.time_ms=6.088 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 180.277138] gitaly[1355]: time="2026-07-18T06:33:47.254Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-18T06:34:17.045" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-18T06:33:47.246" grpc.time_ms=8.304 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689gitlab # [ 180.285452] gitaly[1355]: time="2026-07-18T06:33:47.263Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNEFPZCQQJ4RM7FV81NE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.253" pid=1355 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1690gitlab # [ 180.286165] gitaly[1355]: time="2026-07-18T06:33:47.263Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNEFPZCQQJ4RM7FV81NE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.253" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1691gitlab # [ 180.286777] gitaly[1355]: time="2026-07-18T06:33:47.263Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNEFPZCQQJ4RM7FV81NE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.253" grpc.time_ms=9.61 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1692gitlab # [ 180.288048] gitlab-workhorse[1290]: time="2026-07-18T06:33:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671693gitlab # [ 180.288689] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 481694gitlab # [ 180.289555] gitaly[1355]: time="2026-07-18T06:33:47.266Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T duration_ms=87ns method=POST pid=1355 status=200 url="http://unix/api/v4/internal/allowed"1695gitlab # [ 180.348295] gitaly[1355]: time="2026-07-18T06:33:47.325Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T duration_ms=58ns method=POST pid=1355 status=200 url="http://unix/api/v4/internal/pre_receive"1696gitlab # [ 180.348665] gitaly[1355]: time="2026-07-18T06:33:47.325Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5321 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.178" grpc.time_ms=146.908 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1697gitlab # [ 180.388079] gitaly[1355]: time="2026-07-18T06:33:47.365Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.364" grpc.time_ms=0.366 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1698gitlab # [ 180.424297] gitaly[1355]: time="2026-07-18T06:33:47.401Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNJ1P848WJQXHEAGSFKQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.380" grpc.time_ms=21.427 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1699gitlab # [ 180.427458] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 721700gitlab # [ 180.428507] gitaly[1355]: time="2026-07-18T06:33:47.403Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.401" grpc.time_ms=1.103 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1701gitlab # [ 180.472091] gitaly[1355]: time="2026-07-18T06:33:47.447Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.447" grpc.time_ms=0.202 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1702gitlab # [ 180.475897] gitaly[1355]: time="2026-07-18T06:33:47.453Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNMC7RCBVAGQZYHK610Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.440" pid=1355 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=209 span.kind=server system=grpc1703gitlab # [ 180.476722] gitaly[1355]: time="2026-07-18T06:33:47.453Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNMC7RCBVAGQZYHK610Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.440" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1704gitlab # [ 180.477630] gitaly[1355]: time="2026-07-18T06:33:47.454Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1166 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNMC7RCBVAGQZYHK610Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.440" grpc.time_ms=13.996 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1705gitlab # [ 180.479383] gitlab-workhorse[1290]: time="2026-07-18T06:33:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091706gitlab # [ 180.480065] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.54.0" 501707gitlab # [ 180.515559] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.492Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","87195bc874ea0a4ada07b3774f27b294e19614b7 ce5370efa2035644cca25a7e5dba65a705872d9d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"19e5fe6f6d967fe9e63a474f","created_at":"2026-07-18T06:33:47.490Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-603fead20d9f4180","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.491Z","job_size_bytes":185,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-19e5fe6f6d967fe9e63a474f: start","job_status":"start","queue_duration_s":0.0011,"scheduling_latency_s":0.001129}1708gitlab # [ 180.533640] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.510Z","message":"changes: \"87195bc874ea0a4ada07b3774f27b294e19614b7 ce5370efa2035644cca25a7e5dba65a705872d9d refs/heads/master\\n\""}1709gitlab # [ 180.537606] gitaly[1355]: time="2026-07-18T06:33:47.515Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T duration_ms=42ns method=POST pid=1355 status=200 url="http://unix/api/v4/internal/post_receive"1710gitlab # [ 180.538486] gitaly[1355]: time="2026-07-18T06:33:47.515Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4815 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.472" grpc.time_ms=43.061 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1711gitlab # [ 180.549845] gitaly[1355]: time="2026-07-18T06:33:47.527Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T diskcache=63b09c80-f79a-4fa6-90c7-e87241b0a631 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.127" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1712gitlab # [ 180.552045] gitaly[1355]: time="2026-07-18T06:33:47.527Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=108 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13316 command.oublock=32 command.real_time_ms=389 command.spawn_token_fork_ms=0 command.system_time_ms=72 command.user_time_ms=36 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:47.127" grpc.time_ms=399.695 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1713gitlab # [ 180.552703] gitlab-workhorse[1290]: time="2026-07-18T06:33:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521714gitlab # To http://gitlab/root/test-flake.git1715gitlab # [ 180.553178] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:47 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 5361716gitlab # 87195bc..ce5370e master -> master1717gitlab # [ 180.556250] gitaly[1355]: time="2026-07-18T06:33:47.531Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:57.083" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:47.524" grpc.time_ms=7.159 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1718gitlab # [ 180.565445] gitaly[1355]: time="2026-07-18T06:33:47.543Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-18T06:33:57.097" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:47.538" grpc.time_ms=4.919 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1719gitlab # [ 180.603778] gitaly[1355]: time="2026-07-18T06:33:47.559Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-18T06:33:57.114" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:47.554" grpc.time_ms=4.542 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.77 seconds)1721gitlab: must succeed: git -C /tmp/test-flake rev-parse master1722gitlab # [ 180.625237] gitaly[1355]: time="2026-07-18T06:33:47.602Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-18T06:34:17.596" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.596" grpc.time_ms=6.001 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1723gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1724gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d'1725gitlab # [ 180.651761] gitaly[1355]: time="2026-07-18T06:33:47.629Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-18T06:33:57.183" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-18T06:33:47.624" grpc.time_ms=5.297 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1726gitlab # [ 180.660259] nixbot[2971]: INFO: ::ffff:127.0.0.1:40364 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 200 OK1727gitlab # [ 180.662037] gitaly[1355]: time="2026-07-18T06:33:47.639Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:57.194" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:47.634" grpc.time_ms=4.565 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d', in 0.03 seconds)1729gitlab # [ 180.723550] gitaly[1355]: time="2026-07-18T06:33:47.701Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:17.100" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.700" grpc.time_ms=0.519 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730gitlab # [ 180.761801] gitaly[1355]: time="2026-07-18T06:33:47.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-18T06:34:17.133" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:47.733" grpc.time_ms=5.768 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1731gitlab # [ 180.786498] gitaly[1355]: time="2026-07-18T06:33:47.764Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:57.158" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:47.758" grpc.time_ms=5.179 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1732gitlab # [ 180.827064] gitaly[1355]: time="2026-07-18T06:33:47.804Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-18T06:33:57.204" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:47.804" grpc.time_ms=0.186 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733gitlab # [ 180.845923] gitaly[1355]: time="2026-07-18T06:33:47.823Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-18T06:33:57.218" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:47.818" grpc.time_ms=4.512 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1734gitlab # [ 180.859805] gitaly[1355]: time="2026-07-18T06:33:47.837Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-18T06:33:57.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.836" grpc.time_ms=0.906 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1735gitlab # [ 180.881810] gitaly[1355]: time="2026-07-18T06:33:47.859Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.859" grpc.time_ms=0.28 pid=1355 span.kind=server system=grpc1736gitlab # [ 180.888027] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.865Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","87195bc874ea0a4ada07b3774f27b294e19614b7 ce5370efa2035644cca25a7e5dba65a705872d9d refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"19e5fe6f6d967fe9e63a474f","created_at":"2026-07-18T06:33:47.490Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-603fead20d9f4180","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.491Z","job_size_bytes":185,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-19e5fe6f6d967fe9e63a474f: done: 0.372809 sec","job_status":"done","queue_duration_s":0.0011,"scheduling_latency_s":0.001129,"gitaly_calls":7,"gitaly_duration_s":0.108322,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.020405,"redis_read_bytes":350,"redis_write_bytes":13069,"redis_cache_calls":9,"redis_cache_duration_s":0.001357,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000372,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.000852,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003306,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.00677,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007748,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.204567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050146,"exclusive_lock_hold_duration_s":0.013432993,"duration_s":0.372809,"completed_at":"2026-07-18T06:33:47.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034104,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1737gitlab # [ 180.896301] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.873Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5a84bafd6aa05911bb4e263b","created_at":"2026-07-18T06:33:47.546Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.547Z","job_size_bytes":46,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5a84bafd6aa05911bb4e263b: start","job_status":"start","queue_duration_s":0.325777,"scheduling_latency_s":0.325801}1738gitlab # [ 180.937502] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.886Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5a84bafd6aa05911bb4e263b","created_at":"2026-07-18T06:33:47.546Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.547Z","job_size_bytes":46,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5a84bafd6aa05911bb4e263b: done: 0.0137 sec","job_status":"done","queue_duration_s":0.325777,"scheduling_latency_s":0.325801,"redis_calls":9,"redis_duration_s":0.001188,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000136,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0137,"completed_at":"2026-07-18T06:33:47.886Z","load_balancing_strategy":"primary","db_duration_s":0.001213,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1739gitlab # [ 180.948793] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.888Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"aa1814a2440daa1b71d4aa10","created_at":"2026-07-18T06:33:47.551Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.552Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-aa1814a2440daa1b71d4aa10: start","job_status":"start","queue_duration_s":0.335947,"scheduling_latency_s":0.335961}1740gitlab # [ 180.992765] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.964Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"aa1814a2440daa1b71d4aa10","created_at":"2026-07-18T06:33:47.551Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.552Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-aa1814a2440daa1b71d4aa10: done: 0.076111 sec","job_status":"done","queue_duration_s":0.335947,"scheduling_latency_s":0.335961,"gitaly_calls":1,"gitaly_duration_s":0.055899,"redis_calls":12,"redis_duration_s":0.001821,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000323,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000745,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.064055068,"duration_s":0.076111,"completed_at":"2026-07-18T06:33:47.964Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00298,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1741gitlab # [ 181.032879] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:47.965Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"d666baa52424570970dd0a5b","created_at":"2026-07-18T06:33:47.673Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:011f0d1ac7ca4ccc8435666deda9bca87f2905e6dbe18e5feda44cea60dbf7d7","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.675Z","job_size_bytes":467,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-d666baa52424570970dd0a5b: start","job_status":"start","queue_duration_s":0.290194,"scheduling_latency_s":0.290214}1742gitlab # [ 181.033859] gitaly[1355]: time="2026-07-18T06:33:47.952Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T diskcache=92f2d5e9-3c23-41a3-94bc-3a238e85faad grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-18T12:33:47.897" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.898" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1743gitlab # [ 181.034097] gitaly[1355]: time="2026-07-18T06:33:47.952Z" level=info msg="finished unary call with code OK" catfile.duration_ms=29 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=29 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1647 command.oublock=0 command.real_time_ms=51 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-18T12:33:47.897" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.898" grpc.time_ms=54.347 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 181.075106] gitaly[1355]: time="2026-07-18T06:33:47.958Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:47.957" grpc.time_ms=0.515 pid=1355 span.kind=server system=grpc1745gitlab # [ 181.075736] gitaly[1355]: time="2026-07-18T06:33:47.967Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 diskcache=52721307-e6ae-4940-a97d-c42a3589de22 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-18T06:33:57.232" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:47.833" pid=1355 ref=refs/keep-around/87195bc874ea0a4ada07b3774f27b294e19614b7 remote_ip="2001:db8:1::1" revision=87195bc874ea0a4ada07b3774f27b294e19614b7 span.kind=server system=grpc username=root1746gitlab # [ 181.076406] gitaly[1355]: time="2026-07-18T06:33:47.967Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=114 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5260 command.oublock=8 command.real_time_ms=126 command.spawn_token_fork_ms=0 command.system_time_ms=99 command.user_time_ms=15 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-18T06:33:57.232" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:47.833" grpc.time_ms=134.23 pid=1355 ref=refs/keep-around/87195bc874ea0a4ada07b3774f27b294e19614b7 remote_ip="2001:db8:1::1" revision=87195bc874ea0a4ada07b3774f27b294e19614b7 span.kind=server system=grpc username=root1747gitlab # [ 181.078127] gitaly[1355]: time="2026-07-18T06:33:47.972Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:17.369" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:47.970" grpc.time_ms=2.637 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1748gitlab # [ 181.085188] gitaly[1355]: time="2026-07-18T06:33:48.029Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:18.028" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:48.028" grpc.time_ms=0.666 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1749gitlab # [ 181.097389] gitaly[1355]: time="2026-07-18T06:33:48.035Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-18T06:33:58.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:48.030" grpc.time_ms=4.631 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1750gitlab # [ 181.110741] gitaly[1355]: time="2026-07-18T06:33:48.088Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:18.086" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:48.087" grpc.time_ms=0.85 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751gitlab # [ 181.125338] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/87195bc874ea0a4ada07b3774f27b294e19614b7 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4361752gitlab # [ 181.126075] gitaly[1355]: time="2026-07-18T06:33:48.103Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=228 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-18T06:34:18.093" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:48.094" grpc.time_ms=8.915 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 181.169597] nixbot[2971]: {"ts": "2026-07-18T06:33:48.103Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/87195bc874ea0a4ada07b3774f27b294e19614b7 \"HTTP/1.1 201 Created\""}1754gitlab # [ 181.170451] nixbot[2971]: {"ts": "2026-07-18T06:33:48.109Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-87195bc874ea-aac74a27", "argv0": "bwrap"}1755gitlab # [ 181.208048] gitaly[1355]: time="2026-07-18T06:33:48.185Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:58.159" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:48.179" grpc.time_ms=5.586 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 181.250378] systemd[1]: Started Nix Daemon.1757gitlab # [ 181.289193] nix-daemon[3261]: accepted connection from pid 3260, user nixbot1758gitlab # [ 181.290450] nix-daemon[3261]: accepted connection from pid 3258, user nixbot1759gitlab # [ 181.609432] gitaly[1355]: time="2026-07-18T06:33:48.587Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-18T06:34:18.567" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:48.567" grpc.time_ms=18.947 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d'1761gitlab # [ 181.663295] gitaly[1355]: time="2026-07-18T06:33:48.640Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWN7FKYDGS2CE5SPXJ05T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-18T06:34:18.623" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:48.624" grpc.time_ms=16.518 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 181.687928] sidekiq-cluster[2051]: {"severity":"WARN","time":"2026-07-18T06:33:48.665Z","class":"CreatePipelineWorker","correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"87195bc874ea0a4ada07b3774f27b294e19614b7","after":"ce5370efa2035644cca25a7e5dba65a705872d9d","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ce5370efa2035644cca25a7e5dba65a705872d9d","ref":"refs/heads/master"},"retry":0}1763gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d', in 0.04 seconds)1764gitlab # [ 181.700861] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:48.678Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"d666baa52424570970dd0a5b","created_at":"2026-07-18T06:33:47.673Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:011f0d1ac7ca4ccc8435666deda9bca87f2905e6dbe18e5feda44cea60dbf7d7","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.675Z","job_size_bytes":467,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-d666baa52424570970dd0a5b: done: 0.712406 sec","job_status":"done","queue_duration_s":0.290194,"scheduling_latency_s":0.290214,"gitaly_calls":7,"gitaly_duration_s":0.110494,"redis_calls":33,"redis_duration_s":0.017145,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001288,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.009293,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002818,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.453014,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.712406,"completed_at":"2026-07-18T06:33:48.678Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052731,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1765gitlab # [ 181.714790] nixbot[2971]: INFO: ::ffff:127.0.0.1:40370 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 200 OK1766gitlab # [ 181.746941] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:48.724Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"09f47ae8-c92d-46fa-8d9e-8d1bb505b42a\"}"],"class":"WebHookWorker","jid":"0aab5d5ffafc2e7f3e7b9956","created_at":"2026-07-18T06:33:47.821Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ec2014190f4cdd4f5a39996a664a78fabb22cb1731806cba69584dfc2232db9b","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.826Z","job_size_bytes":1749,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0aab5d5ffafc2e7f3e7b9956: start","job_status":"start","queue_duration_s":0.897727,"scheduling_latency_s":0.897755}1767gitlab # [ 181.918453] nixbot[2971]: INFO: ::ffff:127.0.0.1:40382 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1768gitlab # [ 181.963618] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:48.940Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"09f47ae8-c92d-46fa-8d9e-8d1bb505b42a\"}"],"class":"WebHookWorker","jid":"0aab5d5ffafc2e7f3e7b9956","created_at":"2026-07-18T06:33:47.821Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-55ba36709c854133","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ec2014190f4cdd4f5a39996a664a78fabb22cb1731806cba69584dfc2232db9b","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.826Z","job_size_bytes":1749,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-0aab5d5ffafc2e7f3e7b9956: done: 0.216136 sec","job_status":"done","queue_duration_s":0.897727,"scheduling_latency_s":0.897755,"redis_calls":19,"redis_duration_s":0.021893,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005643,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.001594,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.009315,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005341,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.017911214999998037,"cpu_s":0.101975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001943543,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.216136,"completed_at":"2026-07-18T06:33:48.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1769gitlab # [ 181.968839] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:48.945Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cb492e3702b5bafc15fed968","created_at":"2026-07-18T06:33:47.979Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-1aa1530a35074ea6","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.983Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cb492e3702b5bafc15fed968: start","job_status":"start","queue_duration_s":0.962015,"scheduling_latency_s":0.962128}1770gitlab # [ 182.026911] gitaly[1355]: time="2026-07-18T06:33:49.003Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=10 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQ415TZS1JNYDE5HJEP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:48.989" grpc.time_ms=14.104 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1771gitlab # [ 182.028384] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 751772gitlab # [ 182.085988] gitaly[1355]: time="2026-07-18T06:33:49.063Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQ6VT5Q5S1THSZWA2V5W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.056" pid=1355 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1773gitlab # [ 182.086204] gitaly[1355]: time="2026-07-18T06:33:49.063Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQ6VT5Q5S1THSZWA2V5W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.056" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1774gitlab # [ 182.086853] gitaly[1355]: time="2026-07-18T06:33:49.063Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQ6VT5Q5S1THSZWA2V5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.056" grpc.time_ms=7.237 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1775gitlab # [ 182.087581] gitlab-workhorse[1290]: time="2026-07-18T06:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671776gitlab # [ 182.089158] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 441777gitlab # [ 182.144120] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:49.118Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cb492e3702b5bafc15fed968","created_at":"2026-07-18T06:33:47.979Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-1aa1530a35074ea6","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.983Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cb492e3702b5bafc15fed968: done: 0.173268 sec","job_status":"done","queue_duration_s":0.962015,"scheduling_latency_s":0.962128,"redis_calls":18,"redis_duration_s":0.051039999999999995,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.022514,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.026746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000304508,"duration_s":0.173268,"completed_at":"2026-07-18T06:33:49.118Z","load_balancing_strategy":"primary","db_duration_s":0.060269,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1778gitlab # [ 182.165687] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:49.143Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6f4da22b9e3b2ea2c6eb3526","created_at":"2026-07-18T06:33:47.984Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-1aa1530a35074ea6","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.986Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6f4da22b9e3b2ea2c6eb3526: start","job_status":"start","queue_duration_s":1.156402,"scheduling_latency_s":1.156421}1779gitlab # [ 182.263316] gitaly[1355]: time="2026-07-18T06:33:49.240Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQ890CE84EYZ2BJREB3N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.224" pid=1355 remote_ip="2001:db8:1::1" request_sha=d0eb2c5194c85ef006744578ace66569ae9b5845 response_bytes=337 span.kind=server system=grpc1780gitlab # [ 182.263605] gitaly[1355]: time="2026-07-18T06:33:49.240Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQ890CE84EYZ2BJREB3N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.224" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 182.264243] gitaly[1355]: time="2026-07-18T06:33:49.241Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQ890CE84EYZ2BJREB3N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.224" grpc.time_ms=16.284 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1782gitlab # [ 182.302769] gitlab-workhorse[1290]: time="2026-07-18T06:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371783gitlab # [ 182.303175] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.54.0" 2141784gitlab # [ 182.385244] gitaly[1355]: time="2026-07-18T06:33:49.361Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQGECNHJ2F9BB3Q5C6RY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.356" grpc.time_ms=5.305 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1785gitlab # [ 182.385490] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 361786gitlab # [ 182.409145] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:49.386Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6f4da22b9e3b2ea2c6eb3526","created_at":"2026-07-18T06:33:47.984Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-1aa1530a35074ea6","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:47.986Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6f4da22b9e3b2ea2c6eb3526: done: 0.243131 sec","job_status":"done","queue_duration_s":1.156402,"scheduling_latency_s":1.156421,"redis_calls":15,"redis_duration_s":0.011431,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004104,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.128,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.085,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002863213,"duration_s":0.243131,"completed_at":"2026-07-18T06:33:49.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.167781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1787gitlab # [ 182.455413] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:49.432Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"51cd2eee9c55a988c9d5559e","created_at":"2026-07-18T06:33:48.077Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-1aa1530a35074ea6","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:48.087Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-51cd2eee9c55a988c9d5559e: start","job_status":"start","queue_duration_s":1.345105,"scheduling_latency_s":1.345128}1788gitlab # [ 182.491113] gitaly[1355]: time="2026-07-18T06:33:49.468Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQHKEYRKSDE4542X81DV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.456" pid=1355 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc1789gitlab # [ 182.491875] gitaly[1355]: time="2026-07-18T06:33:49.468Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQHKEYRKSDE4542X81DV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.456" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1790gitlab # [ 182.492829] gitlab-workhorse[1290]: time="2026-07-18T06:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161791gitlab # [ 182.493617] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 1051792gitlab # [ 182.496148] gitaly[1355]: time="2026-07-18T06:33:49.468Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQHKEYRKSDE4542X81DV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:33:49.456" grpc.time_ms=12.561 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 182.564196] nixbot[2971]: {"ts": "2026-07-18T06:33:49.541Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}1794gitlab # [ 182.571500] nix-daemon[3267]: unexpected Nix daemon error: error: interrupted by the user1795gitlab # [ 182.594542] gitaly[1355]: time="2026-07-18T06:33:49.572Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:19.571" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.571" grpc.time_ms=0.698 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1796gitlab # [ 182.627689] gitaly[1355]: time="2026-07-18T06:33:49.605Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.604" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.604" grpc.time_ms=0.405 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab # [ 182.629194] gitaly[1355]: time="2026-07-18T06:33:49.606Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.405" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.606" grpc.time_ms=0.348 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 182.648532] gitaly[1355]: time="2026-07-18T06:33:49.626Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-18T06:34:19.420" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:49.620" grpc.time_ms=5.205 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799gitlab # [ 182.671818] gitaly[1355]: time="2026-07-18T06:33:49.649Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:59.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:49.644" grpc.time_ms=4.754 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 182.705059] gitaly[1355]: time="2026-07-18T06:33:49.682Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-18T06:33:59.042" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:49.682" grpc.time_ms=0.186 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d'1802gitlab # [ 182.715152] gitaly[1355]: time="2026-07-18T06:33:49.692Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-18T06:33:59.045" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:49.685" grpc.time_ms=6.718 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab # [ 182.727968] gitaly[1355]: time="2026-07-18T06:33:49.705Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWNWHMVZ430T50QHR5SA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.704" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.704" grpc.time_ms=0.392 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 182.748185] nixbot[2971]: INFO: ::ffff:127.0.0.1:40390 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 200 OK1805gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ce5370efa2035644cca25a7e5dba65a705872d9d', in 0.03 seconds)1806(finished: subtest: push delivers the webhook, in 2.90 seconds)1807subtest: eval and build post commit statuses1808gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1809gitlab # [ 182.756805] gitaly[1355]: time="2026-07-18T06:33:49.734Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:49.732" grpc.time_ms=1.221 pid=1355 span.kind=server system=grpc1810gitlab # [ 182.770674] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:49.747Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"51cd2eee9c55a988c9d5559e","created_at":"2026-07-18T06:33:48.077Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-1aa1530a35074ea6","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:48.087Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-51cd2eee9c55a988c9d5559e: done: 0.314896 sec","job_status":"done","queue_duration_s":1.345105,"scheduling_latency_s":1.345128,"gitaly_calls":3,"gitaly_duration_s":0.007996,"redis_calls":50,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.032851,"redis_read_bytes":210,"redis_write_bytes":7083,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.006485,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002322,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.010337,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1275,"redis_queues_calls":2,"redis_queues_duration_s":0.000988,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002138,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001641,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000659,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001796,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":35,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.154,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.138,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.157599,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.224830809,"duration_s":0.314896,"completed_at":"2026-07-18T06:33:49.747Z","load_balancing_strategy":"primary","db_duration_s":0.163224,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1811gitlab # [ 182.812120] gitaly[1355]: time="2026-07-18T06:33:49.751Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.151" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.751" grpc.time_ms=0.341 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 182.812738] gitaly[1355]: time="2026-07-18T06:33:49.756Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4160 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:49.756" grpc.time_ms=0.24 pid=1355 span.kind=server system=grpc1813gitlab # [ 182.844075] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:49.821Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0aab5d5ffafc2e7f3e7b9956"],"class":"WebHooks::LogExecutionWorker","jid":"a822d3b53b6d421089da75d1","created_at":"2026-07-18T06:33:48.905Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-01f430ddb443401e","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c07574431ee4ec086376929cea22eceb268fda02f6e823bed190e787f4c563a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:48.918Z","job_size_bytes":2507,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a822d3b53b6d421089da75d1: start","job_status":"start","queue_duration_s":0.902157,"scheduling_latency_s":0.902183}1814gitlab # [ 182.847263] gitaly[1355]: time="2026-07-18T06:33:49.824Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ diskcache=4800b249-3a5c-46d4-a227-38f4018ae092 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-18T06:33:59.056" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:49.696" pid=1355 ref=refs/keep-around/ce5370efa2035644cca25a7e5dba65a705872d9d remote_ip="2001:db8:1::1" revision=ce5370efa2035644cca25a7e5dba65a705872d9d span.kind=server system=grpc username=root1815gitlab # [ 182.847884] gitaly[1355]: time="2026-07-18T06:33:49.824Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=96 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5087 command.oublock=8 command.real_time_ms=116 command.spawn_token_fork_ms=0 command.system_time_ms=79 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-18T06:33:59.056" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-18T06:33:49.696" grpc.time_ms=128.314 pid=1355 ref=refs/keep-around/ce5370efa2035644cca25a7e5dba65a705872d9d remote_ip="2001:db8:1::1" revision=ce5370efa2035644cca25a7e5dba65a705872d9d span.kind=server system=grpc username=root1816gitlab # [ 182.851792] gitaly[1355]: time="2026-07-18T06:33:49.829Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:33:59.221" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:49.822" grpc.time_ms=6.985 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 182.867392] gitaly[1355]: time="2026-07-18T06:33:49.844Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.844" grpc.time_ms=0.301 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 182.938518] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/87195bc874ea0a4ada07b3774f27b294e19614b7 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 2531819gitlab # [ 182.940681] nixbot[2971]: {"ts": "2026-07-18T06:33:49.917Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/87195bc874ea0a4ada07b3774f27b294e19614b7 \"HTTP/1.1 201 Created\""}1820gitlab # [ 182.961675] gitaly[1355]: time="2026-07-18T06:33:49.939Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQZRJHFFRQQG2PZ5C75T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.238" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.938" grpc.time_ms=0.422 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 182.991026] gitaly[1355]: time="2026-07-18T06:33:49.968Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWR304RCKENZTX2DXQN1N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:19.367" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:49.968" grpc.time_ms=0.404 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 183.000411] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4331823gitlab # [ 183.003094] nixbot[2971]: {"ts": "2026-07-18T06:33:49.978Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d \"HTTP/1.1 201 Created\""}1824gitlab # [ 183.008204] nixbot[2971]: {"ts": "2026-07-18T06:33:49.984Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ce5370efa203-d4822045", "argv0": "bwrap"}1825gitlab # [ 183.016126] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1771826gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.27 seconds)1827gitlab # [ 183.025108] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.002Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","0aab5d5ffafc2e7f3e7b9956"],"class":"WebHooks::LogExecutionWorker","jid":"a822d3b53b6d421089da75d1","created_at":"2026-07-18T06:33:48.905Z","trace_propagation_headers":{"sentry-trace":"bdd61ded6b324d2ebb05b6ed5205b1e5-01f430ddb443401e","baggage":"sentry-trace_id=bdd61ded6b324d2ebb05b6ed5205b1e5,sentry-release=Unknown"},"correlation_id":"01KXSYWN7FKYDGS2CE5SPXJ05T","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c07574431ee4ec086376929cea22eceb268fda02f6e823bed190e787f4c563a","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:48.918Z","job_size_bytes":2507,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a822d3b53b6d421089da75d1: done: 0.181381 sec","job_status":"done","queue_duration_s":0.902157,"scheduling_latency_s":0.902183,"redis_calls":18,"redis_duration_s":0.011402,"redis_read_bytes":525,"redis_write_bytes":1835,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00706,"redis_feature_flag_read_bytes":518,"redis_feature_flag_write_bytes":1089,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004047,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000295,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.038,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.038,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214273,"duration_s":0.181381,"completed_at":"2026-07-18T06:33:50.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1828gitlab # [ 183.066159] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c45a90d0cc19117a44352caa","created_at":"2026-07-18T06:33:49.540Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-a29d9e0756e64011","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.554Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c45a90d0cc19117a44352caa: start","job_status":"start","queue_duration_s":0.459361,"scheduling_latency_s":0.459379}1829gitlab # [ 183.080410] gitaly[1355]: time="2026-07-18T06:33:50.034Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWR304RCKENZTX2DXQN1N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:34:00.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:50.028" grpc.time_ms=5.895 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1830gitlab # [ 183.093541] nix-daemon[3261]: accepted connection from pid 3372, user nixbot1831gitlab # [ 183.094540] nix-daemon[3261]: accepted connection from pid 3370, user nixbot1832gitlab # [ 183.098464] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/87195bc874ea0a4ada07b3774f27b294e19614b7 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1541833gitlab # [ 183.098804] nixbot[2971]: {"ts": "2026-07-18T06:33:50.075Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/87195bc874ea0a4ada07b3774f27b294e19614b7 \"HTTP/1.1 201 Created\""}1834gitlab # [ 183.214456] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c45a90d0cc19117a44352caa","created_at":"2026-07-18T06:33:49.540Z","trace_propagation_headers":{"sentry-trace":"3ee0fb34dd2842559506c154e24e8448-a29d9e0756e64011","baggage":"sentry-trace_id=3ee0fb34dd2842559506c154e24e8448,sentry-release=Unknown"},"correlation_id":"01KXSYWNWHMVZ430T50QHR5SA9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.554Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c45a90d0cc19117a44352caa: done: 0.177136 sec","job_status":"done","queue_duration_s":0.459361,"scheduling_latency_s":0.459379,"redis_calls":10,"redis_duration_s":0.008765,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00341,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000672,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.127,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.116,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002296381,"duration_s":0.177136,"completed_at":"2026-07-18T06:33:50.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.126887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1835gitlab # [ 183.287859] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.264Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ebb9466c2eacbc203b7c9d6e","created_at":"2026-07-18T06:33:49.847Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-b641064242704670","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.851Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ebb9466c2eacbc203b7c9d6e: start","job_status":"start","queue_duration_s":0.413492,"scheduling_latency_s":0.413543}1836gitlab # [ 183.356259] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.333Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ebb9466c2eacbc203b7c9d6e","created_at":"2026-07-18T06:33:49.847Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-b641064242704670","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.851Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ebb9466c2eacbc203b7c9d6e: done: 0.068311 sec","job_status":"done","queue_duration_s":0.413492,"scheduling_latency_s":0.413543,"redis_calls":7,"redis_duration_s":0.006999,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001576,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068311,"completed_at":"2026-07-18T06:33:50.332Z","load_balancing_strategy":"primary","db_duration_s":0.007369,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1837gitlab # [ 183.407463] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.384Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f70bfee7e362f087b93a6ea3","created_at":"2026-07-18T06:33:49.852Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-b641064242704670","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.855Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f70bfee7e362f087b93a6ea3: start","job_status":"start","queue_duration_s":0.529221,"scheduling_latency_s":0.529268}1838gitlab # [ 183.530511] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.507Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f70bfee7e362f087b93a6ea3","created_at":"2026-07-18T06:33:49.852Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-b641064242704670","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.855Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f70bfee7e362f087b93a6ea3: done: 0.122958 sec","job_status":"done","queue_duration_s":0.529221,"scheduling_latency_s":0.529268,"redis_calls":10,"redis_duration_s":0.013616,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.007193,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.122958,"completed_at":"2026-07-18T06:33:50.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0347,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1839gitlab # [ 183.597480] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:50.574Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"274eb9bb4baf71a0168699d7","created_at":"2026-07-18T06:33:49.905Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-ccb169cf34b2495f","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.907Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-274eb9bb4baf71a0168699d7: start","job_status":"start","queue_duration_s":0.667053,"scheduling_latency_s":0.667107}1840gitlab # [ 183.810872] gitaly[1355]: time="2026-07-18T06:33:50.786Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:50.784" grpc.time_ms=0.76 pid=1355 span.kind=server system=grpc1841gitlab # [ 183.855191] gitaly[1355]: time="2026-07-18T06:33:50.832Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:50.832" grpc.time_ms=0.397 pid=1355 span.kind=server system=grpc1842gitlab # [ 183.885477] gitaly[1355]: time="2026-07-18T06:33:50.863Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:50.862" grpc.time_ms=0.226 pid=1355 span.kind=server system=grpc1843gitlab # [ 183.903986] gitaly[1355]: time="2026-07-18T06:33:50.881Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 diskcache=cf49e665-f1f1-4ee5-a247-d6d4425b8c82 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:20.743" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:50.744" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1844gitlab # [ 183.904535] gitaly[1355]: time="2026-07-18T06:33:50.881Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=119 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7158 command.oublock=0 command.real_time_ms=128 command.spawn_token_fork_ms=0 command.system_time_ms=84 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:20.743" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:50.744" grpc.time_ms=137.347 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1845gitlab # [ 183.952811] gitaly[1355]: time="2026-07-18T06:33:50.930Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:20.029" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:50.929" grpc.time_ms=0.871 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1846gitlab # [ 183.986266] gitaly[1355]: time="2026-07-18T06:33:50.963Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:20.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:50.962" grpc.time_ms=0.921 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1847gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1848gitlab # [ 184.034651] gitaly[1355]: time="2026-07-18T06:33:51.012Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQTYHNE7G9YBY8T6JSC5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:21.009" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:51.010" grpc.time_ms=1.556 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849gitlab # [ 184.049737] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.024Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"274eb9bb4baf71a0168699d7","created_at":"2026-07-18T06:33:49.905Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-ccb169cf34b2495f","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.907Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2051,"job_status":"rescheduled","message":"PipelineProcessWorker JID-274eb9bb4baf71a0168699d7: rescheduled"}1850gitlab # [ 184.061082] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.037Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"274eb9bb4baf71a0168699d7","created_at":"2026-07-18T06:33:49.905Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-ccb169cf34b2495f","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.907Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-274eb9bb4baf71a0168699d7: done: 0.463463 sec","job_status":"done","queue_duration_s":0.667053,"scheduling_latency_s":0.667107,"gitaly_calls":4,"gitaly_duration_s":0.178068,"redis_calls":101,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.044544,"redis_read_bytes":185,"redis_write_bytes":23555,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00613,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002119,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.010862,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.004021,"redis_queues_read_bytes":25,"redis_queues_write_bytes":12387,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.010552,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000734,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000317,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003679,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.023,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.170323,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.401008788,"duration_s":0.463463,"completed_at":"2026-07-18T06:33:51.037Z","load_balancing_strategy":"primary","db_duration_s":0.077313,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1851gitlab # [ 184.082542] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.043Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"650441466e7e45cfba610d2f","created_at":"2026-07-18T06:33:49.961Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-b641064242704670","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.964Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-650441466e7e45cfba610d2f: start","job_status":"start","queue_duration_s":1.078173,"scheduling_latency_s":1.078202}1852gitlab # [ 184.156857] gitaly[1355]: time="2026-07-18T06:33:51.134Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:21.133" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:51.133" grpc.time_ms=0.76 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853gitlab # [ 184.168411] gitaly[1355]: time="2026-07-18T06:33:51.143Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWS5SX1F9TXPMR92H4823 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:21.041" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:51.142" grpc.time_ms=0.838 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1854gitlab # [ 184.176362] gitaly[1355]: time="2026-07-18T06:33:51.154Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:21.153" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:51.153" grpc.time_ms=0.413 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 184.207035] gitaly[1355]: time="2026-07-18T06:33:51.184Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWQQ75EN5KNSJ3FM3AVWQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:21.183" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:51.184" grpc.time_ms=0.392 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 184.211205] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1541857gitlab # [ 184.213981] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.191Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"650441466e7e45cfba610d2f","created_at":"2026-07-18T06:33:49.961Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-b641064242704670","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:49.964Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-650441466e7e45cfba610d2f: done: 0.148462 sec","job_status":"done","queue_duration_s":1.078173,"scheduling_latency_s":1.078202,"gitaly_calls":3,"gitaly_duration_s":0.006233,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007547,"redis_read_bytes":99,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00107,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001626,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.00028,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001872,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000251,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000572,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000806,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094206,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.107080369,"duration_s":0.148462,"completed_at":"2026-07-18T06:33:51.191Z","load_balancing_strategy":"primary","db_duration_s":0.04029,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1858gitlab # [ 184.226605] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f8357255e5b188aeea3b2b61","created_at":"2026-07-18T06:33:50.707Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.720Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f8357255e5b188aeea3b2b61: start","job_status":"start","queue_duration_s":0.471903,"scheduling_latency_s":0.471919}1859gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.26 seconds)1860gitlab # [ 184.287542] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f8357255e5b188aeea3b2b61","created_at":"2026-07-18T06:33:50.707Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.720Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f8357255e5b188aeea3b2b61: done: 0.072534 sec","job_status":"done","queue_duration_s":0.471903,"scheduling_latency_s":0.471919,"redis_calls":15,"redis_duration_s":0.001988,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000195,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000317,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052123,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000193321,"duration_s":0.072534,"completed_at":"2026-07-18T06:33:51.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015947,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1861gitlab # [ 184.327801] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.266Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"e1e22e82dcda5e743d379ae4","created_at":"2026-07-18T06:33:51.258Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-8ae071ead723453b","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:539825966580cd5645670879621526352b3218bbd4011346873c74d8b5d90be7","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:51.260Z","job_size_bytes":464,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-e1e22e82dcda5e743d379ae4: start","job_status":"start","queue_duration_s":0.005699,"scheduling_latency_s":0.005714}1862gitlab # [ 184.372249] sidekiq-cluster[2051]: {"severity":"WARN","time":"2026-07-18T06:33:51.349Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"e1e22e82dcda5e743d379ae4","created_at":"2026-07-18T06:33:51.258Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-8ae071ead723453b","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:539825966580cd5645670879621526352b3218bbd4011346873c74d8b5d90be7","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:51.260Z","job_size_bytes":464,"pid":2051,"sidekiq_tid":"m31f","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-e1e22e82dcda5e743d379ae4: fail: 0.083112 sec","job_status":"fail","queue_duration_s":0.005699,"scheduling_latency_s":0.005714,"redis_calls":7,"redis_duration_s":0.0013939999999999998,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083112,"completed_at":"2026-07-18T06:33:51.349Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.003848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1863gitlab # [ 184.707757] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.352Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"af1c225a94ccad833ec1eeeb","created_at":"2026-07-18T06:33:50.886Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.889Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-af1c225a94ccad833ec1eeeb: start","job_status":"start","queue_duration_s":0.463013,"scheduling_latency_s":0.463064}1864gitlab # [ 184.708687] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.453Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"af1c225a94ccad833ec1eeeb","created_at":"2026-07-18T06:33:50.886Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.889Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-af1c225a94ccad833ec1eeeb: done: 0.101256 sec","job_status":"done","queue_duration_s":0.463013,"scheduling_latency_s":0.463064,"redis_calls":10,"redis_duration_s":0.002386,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001397,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402006,"duration_s":0.101256,"completed_at":"2026-07-18T06:33:51.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865gitlab # [ 184.726927] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.455Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"006dd1078240dc24e3f230cf","created_at":"2026-07-18T06:33:50.897Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.902Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-006dd1078240dc24e3f230cf: start","job_status":"start","queue_duration_s":0.552987,"scheduling_latency_s":0.553019}1866gitlab # [ 184.769988] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.529Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"006dd1078240dc24e3f230cf","created_at":"2026-07-18T06:33:50.897Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.902Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-006dd1078240dc24e3f230cf: done: 0.07418 sec","job_status":"done","queue_duration_s":0.552987,"scheduling_latency_s":0.553019,"redis_calls":10,"redis_duration_s":0.0023179999999999997,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000335,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.038,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000401169,"duration_s":0.07418,"completed_at":"2026-07-18T06:33:51.529Z","load_balancing_strategy":"primary","db_duration_s":0.038078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867gitlab # [ 184.817393] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.531Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"7c7b60fde52507763009b87b","created_at":"2026-07-18T06:33:50.911Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.913Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7c7b60fde52507763009b87b: start","job_status":"start","queue_duration_s":0.617598,"scheduling_latency_s":0.617631}1868gitlab # [ 184.818220] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"7c7b60fde52507763009b87b","created_at":"2026-07-18T06:33:50.911Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.913Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7c7b60fde52507763009b87b: done: 0.035282 sec","job_status":"done","queue_duration_s":0.617598,"scheduling_latency_s":0.617631,"redis_calls":13,"redis_duration_s":0.003919,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000379,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000747,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000664,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000956546,"duration_s":0.035282,"completed_at":"2026-07-18T06:33:51.566Z","load_balancing_strategy":"primary","db_duration_s":0.003988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1869gitlab # [ 184.871449] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.567Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4aff1aea7c7ccf1bca0193eb","created_at":"2026-07-18T06:33:50.914Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.916Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4aff1aea7c7ccf1bca0193eb: start","job_status":"start","queue_duration_s":0.651561,"scheduling_latency_s":0.651576}1870gitlab # [ 184.872552] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.587Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4aff1aea7c7ccf1bca0193eb","created_at":"2026-07-18T06:33:50.914Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.916Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4aff1aea7c7ccf1bca0193eb: done: 0.019288 sec","job_status":"done","queue_duration_s":0.651561,"scheduling_latency_s":0.651576,"redis_calls":10,"redis_duration_s":0.0021479999999999997,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001261,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000272381,"duration_s":0.019288,"completed_at":"2026-07-18T06:33:51.587Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004861,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1871gitlab # [ 184.911418] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.588Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"bf149d8d66a76ac2934bc92e","created_at":"2026-07-18T06:33:50.916Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.919Z","job_size_bytes":19,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-bf149d8d66a76ac2934bc92e: start","job_status":"start","queue_duration_s":0.668951,"scheduling_latency_s":0.668964}1872gitlab # [ 184.912538] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.608Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"bf149d8d66a76ac2934bc92e","created_at":"2026-07-18T06:33:50.916Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.919Z","job_size_bytes":19,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-bf149d8d66a76ac2934bc92e: done: 0.020058 sec","job_status":"done","queue_duration_s":0.668951,"scheduling_latency_s":0.668964,"redis_calls":11,"redis_duration_s":0.002372,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001512,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000308978,"duration_s":0.020058,"completed_at":"2026-07-18T06:33:51.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00464,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873gitlab # [ 184.941679] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.609Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c5748336c93ac4694410693c","created_at":"2026-07-18T06:33:50.921Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.923Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c5748336c93ac4694410693c: start","job_status":"start","queue_duration_s":0.68598,"scheduling_latency_s":0.685994}1874gitlab # [ 184.957497] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.640Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c5748336c93ac4694410693c","created_at":"2026-07-18T06:33:50.921Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.923Z","job_size_bytes":65,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c5748336c93ac4694410693c: done: 0.031023 sec","job_status":"done","queue_duration_s":0.68598,"scheduling_latency_s":0.685994,"redis_calls":10,"redis_duration_s":0.002504,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001581,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000443912,"duration_s":0.031023,"completed_at":"2026-07-18T06:33:51.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1875gitlab # [ 184.996773] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.642Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b6a28317767cb8cdba5cdf31","created_at":"2026-07-18T06:33:50.924Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.925Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b6a28317767cb8cdba5cdf31: start","job_status":"start","queue_duration_s":0.716865,"scheduling_latency_s":0.716892}1876gitlab # [ 185.044488] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.665Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"b6a28317767cb8cdba5cdf31","created_at":"2026-07-18T06:33:50.924Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.925Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b6a28317767cb8cdba5cdf31: done: 0.023633 sec","job_status":"done","queue_duration_s":0.716865,"scheduling_latency_s":0.716892,"redis_calls":7,"redis_duration_s":0.001631,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023633,"completed_at":"2026-07-18T06:33:51.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003507,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1877gitlab # [ 185.046900] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.666Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2b34119b20f9efe2eb3c4ecd","created_at":"2026-07-18T06:33:50.980Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.984Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2b34119b20f9efe2eb3c4ecd: start","job_status":"start","queue_duration_s":0.68262,"scheduling_latency_s":0.682638}1878gitlab # [ 185.063474] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.697Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2b34119b20f9efe2eb3c4ecd","created_at":"2026-07-18T06:33:50.980Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.984Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2b34119b20f9efe2eb3c4ecd: done: 0.030387 sec","job_status":"done","queue_duration_s":0.68262,"scheduling_latency_s":0.682638,"redis_calls":10,"redis_duration_s":0.0022240000000000003,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001476,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000265118,"duration_s":0.030387,"completed_at":"2026-07-18T06:33:51.697Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1879gitlab # [ 185.116684] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"53da0e6e4bb53cfc971a65ab","created_at":"2026-07-18T06:33:50.984Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.987Z","job_size_bytes":6,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-53da0e6e4bb53cfc971a65ab: start","job_status":"start","queue_duration_s":0.711343,"scheduling_latency_s":0.711374}1880gitlab # [ 185.117548] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"53da0e6e4bb53cfc971a65ab","created_at":"2026-07-18T06:33:50.984Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:50.987Z","job_size_bytes":6,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-53da0e6e4bb53cfc971a65ab: done: 0.018382 sec","job_status":"done","queue_duration_s":0.711343,"scheduling_latency_s":0.711374,"redis_calls":10,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000242209,"duration_s":0.018382,"completed_at":"2026-07-18T06:33:51.717Z","load_balancing_strategy":"primary","db_duration_s":0.001685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1881gitlab # [ 185.168604] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.718Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c9a2e4470b1047891a312a97","created_at":"2026-07-18T06:33:51.025Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:51.030Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c9a2e4470b1047891a312a97: start","job_status":"start","queue_duration_s":0.687843,"scheduling_latency_s":0.687856}1882gitlab # [ 185.169465] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.729Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c9a2e4470b1047891a312a97","created_at":"2026-07-18T06:33:51.025Z","trace_propagation_headers":{"sentry-trace":"96eb9876690b4050a80a1cae04d83695-3e4d244bf7834720","baggage":"sentry-trace_id=96eb9876690b4050a80a1cae04d83695,sentry-release=Unknown"},"correlation_id":"01KXSYWQTYHNE7G9YBY8T6JSC5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:51.030Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c9a2e4470b1047891a312a97: done: 0.011742 sec","job_status":"done","queue_duration_s":0.687843,"scheduling_latency_s":0.687856,"redis_calls":8,"redis_duration_s":0.001488,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001106,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011742,"completed_at":"2026-07-18T06:33:51.729Z","load_balancing_strategy":"primary","db_duration_s":0.00169,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1883gitlab # [ 185.211890] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a8a2bacf3a4b66d831989da7","created_at":"2026-07-18T06:33:51.122Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-fed1293c06f74c05","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:51.125Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a8a2bacf3a4b66d831989da7: start","job_status":"start","queue_duration_s":0.605847,"scheduling_latency_s":0.60586}1884gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1885gitlab # [ 185.217764] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:51.748Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a8a2bacf3a4b66d831989da7","created_at":"2026-07-18T06:33:51.122Z","trace_propagation_headers":{"sentry-trace":"b896e7e68c3c487282bd667ecf66398f-fed1293c06f74c05","baggage":"sentry-trace_id=b896e7e68c3c487282bd667ecf66398f,sentry-release=Unknown"},"correlation_id":"01KXSYWQQ75EN5KNSJ3FM3AVWQ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:51.125Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a8a2bacf3a4b66d831989da7: done: 0.017715 sec","job_status":"done","queue_duration_s":0.605847,"scheduling_latency_s":0.60586,"redis_calls":7,"redis_duration_s":0.001487,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017715,"completed_at":"2026-07-18T06:33:51.748Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1886gitlab # [ 185.432578] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1023138904 wd_nsec: 10231388321887gitlab # [ 185.397275] gitaly[1355]: time="2026-07-18T06:33:52.374Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWTDSFJVDMY6QXMVQE9WV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:22.172" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:52.372" grpc.time_ms=0.904 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 185.437887] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1011889gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.16 seconds)1890gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1891gitlab # [ 186.535703] gitaly[1355]: time="2026-07-18T06:33:53.512Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWVHKSH35J7C024NV0JH3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:23.410" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:53.511" grpc.time_ms=0.882 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 186.564752] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 821893gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.12 seconds)1894gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1895gitlab # [ 187.660398] gitaly[1355]: time="2026-07-18T06:33:54.637Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWMME6EQA4GCQJSJGBAX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:24.435" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:54.636" grpc.time_ms=0.914 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1896gitlab # [ 187.684726] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 811897gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.12 seconds)1898gitlab # [ 187.832916] nix-daemon[3261]: accepted connection from pid 3433, user nixbot1899gitlab # [ 187.859360] gitaly[1355]: time="2026-07-18T06:33:54.836Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:24.235" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:54.835" grpc.time_ms=0.789 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 187.888554] gitaly[1355]: time="2026-07-18T06:33:54.866Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:34:04.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:54.857" grpc.time_ms=8.71 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 187.914803] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:54.891Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f4632a0f1cbd658152c64bbb","created_at":"2026-07-18T06:33:54.886Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-6a6b97468c394a37","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:54.888Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f4632a0f1cbd658152c64bbb: start","job_status":"start","queue_duration_s":0.002233,"scheduling_latency_s":0.002283}1902gitlab # [ 187.920745] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 1061903gitlab # [ 187.922555] nixbot[2971]: {"ts": "2026-07-18T06:33:54.899Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d \"HTTP/1.1 201 Created\""}1904gitlab # [ 187.974087] gitaly[1355]: time="2026-07-18T06:33:54.951Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWYSVRTYQX05QC87K0JS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:24.350" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:54.950" grpc.time_ms=0.689 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 188.008656] gitaly[1355]: time="2026-07-18T06:33:54.986Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWYSVRTYQX05QC87K0JS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:34:04.030" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:54.980" grpc.time_ms=5.283 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 188.028691] gitaly[1355]: time="2026-07-18T06:33:55.006Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:55.005" grpc.time_ms=1.103 pid=1355 span.kind=server system=grpc1907gitlab # [ 188.037023] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1091908gitlab # [ 188.037866] nixbot[2971]: {"ts": "2026-07-18T06:33:55.015Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d \"HTTP/1.1 201 Created\""}1909gitlab # [ 188.060357] gitaly[1355]: time="2026-07-18T06:33:55.038Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:55.037" grpc.time_ms=0.348 pid=1355 span.kind=server system=grpc1910gitlab # [ 188.087543] gitaly[1355]: time="2026-07-18T06:33:55.065Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:33:55.064" grpc.time_ms=0.276 pid=1355 span.kind=server system=grpc1911gitlab # [ 188.105160] gitaly[1355]: time="2026-07-18T06:33:55.082Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 diskcache=76e5f6a1-1fa9-49a9-9588-967a4e96e0c6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:24.076" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:54.977" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 188.113985] gitaly[1355]: time="2026-07-18T06:33:55.091Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7351 command.oublock=0 command.real_time_ms=98 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:24.076" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:54.977" grpc.time_ms=105.907 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 188.156946] gitaly[1355]: time="2026-07-18T06:33:55.134Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:25.133" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.133" grpc.time_ms=0.614 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 188.178847] gitaly[1355]: time="2026-07-18T06:33:55.156Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:25.154" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.155" grpc.time_ms=0.708 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 188.220352] gitaly[1355]: time="2026-07-18T06:33:55.197Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:25.196" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.197" grpc.time_ms=0.598 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 188.231814] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.209Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7d566734c9c51020510a8400","created_at":"2026-07-18T06:33:55.205Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"f4632a0f1cbd658152c64bbb","job_size_bytes":3,"pid":2051,"job_status":"deduplicated","message":"PipelineProcessWorker JID-7d566734c9c51020510a8400: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1917gitlab # [ 188.234311] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.211Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f4632a0f1cbd658152c64bbb","created_at":"2026-07-18T06:33:54.886Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-6a6b97468c394a37","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:54.888Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2051,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f4632a0f1cbd658152c64bbb: rescheduled"}1918gitlab # [ 188.243991] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.221Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f4632a0f1cbd658152c64bbb","created_at":"2026-07-18T06:33:54.886Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-6a6b97468c394a37","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:54.888Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f4632a0f1cbd658152c64bbb: done: 0.330057 sec","job_status":"done","queue_duration_s":0.002233,"scheduling_latency_s":0.002283,"gitaly_calls":4,"gitaly_duration_s":0.134774,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.019607,"redis_read_bytes":262,"redis_write_bytes":23388,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001205,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002313,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001182,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.00365,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.007174,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000328,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000715,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001835,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140149,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.287090678,"duration_s":0.330057,"completed_at":"2026-07-18T06:33:55.221Z","load_balancing_strategy":"primary","db_duration_s":0.039605,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1919gitlab # [ 188.267571] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.223Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c24cf3dd9c60a4a5eaed2d60","created_at":"2026-07-18T06:33:54.971Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:54.973Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c24cf3dd9c60a4a5eaed2d60: start","job_status":"start","queue_duration_s":0.249593,"scheduling_latency_s":0.249628}1920gitlab # [ 188.317865] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.262Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c24cf3dd9c60a4a5eaed2d60","created_at":"2026-07-18T06:33:54.971Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:54.973Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c24cf3dd9c60a4a5eaed2d60: done: 0.039664 sec","job_status":"done","queue_duration_s":0.249593,"scheduling_latency_s":0.249628,"redis_calls":7,"redis_duration_s":0.001951,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039664,"completed_at":"2026-07-18T06:33:55.262Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007693,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1921gitlab # [ 188.320726] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.264Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f3e55c3bd4a0ecd4169cec3d","created_at":"2026-07-18T06:33:55.106Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.109Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f3e55c3bd4a0ecd4169cec3d: start","job_status":"start","queue_duration_s":0.154804,"scheduling_latency_s":0.15482}1922gitlab # [ 188.335687] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.279Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f3e55c3bd4a0ecd4169cec3d","created_at":"2026-07-18T06:33:55.106Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.109Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f3e55c3bd4a0ecd4169cec3d: done: 0.015582 sec","job_status":"done","queue_duration_s":0.154804,"scheduling_latency_s":0.15482,"redis_calls":7,"redis_duration_s":0.001479,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015582,"completed_at":"2026-07-18T06:33:55.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923gitlab # [ 188.377341] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.281Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0fce5e642296aa8984f80483","created_at":"2026-07-18T06:33:55.114Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.115Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0fce5e642296aa8984f80483: start","job_status":"start","queue_duration_s":0.165101,"scheduling_latency_s":0.165115}1924gitlab # [ 188.422801] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.377Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"0fce5e642296aa8984f80483","created_at":"2026-07-18T06:33:55.114Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.115Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-0fce5e642296aa8984f80483: done: 0.096446 sec","job_status":"done","queue_duration_s":0.165101,"scheduling_latency_s":0.165115,"redis_calls":7,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.075,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.096446,"completed_at":"2026-07-18T06:33:55.377Z","load_balancing_strategy":"primary","db_duration_s":0.074802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1925gitlab # [ 188.424815] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"278b09120559f9d49d74f53f","created_at":"2026-07-18T06:33:55.121Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.122Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-278b09120559f9d49d74f53f: start","job_status":"start","queue_duration_s":0.256639,"scheduling_latency_s":0.256677}1926gitlab # [ 188.466183] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.411Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"278b09120559f9d49d74f53f","created_at":"2026-07-18T06:33:55.121Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.122Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-278b09120559f9d49d74f53f: done: 0.032815 sec","job_status":"done","queue_duration_s":0.256639,"scheduling_latency_s":0.256677,"redis_calls":10,"redis_duration_s":0.002284,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000527,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032815,"completed_at":"2026-07-18T06:33:55.411Z","load_balancing_strategy":"primary","db_duration_s":0.002609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1927gitlab # [ 188.478365] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ab9027d93240c168e8ce4c05","created_at":"2026-07-18T06:33:55.122Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.123Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ab9027d93240c168e8ce4c05: start","job_status":"start","queue_duration_s":0.29017,"scheduling_latency_s":0.290209}1928gitlab # [ 188.524701] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.440Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ab9027d93240c168e8ce4c05","created_at":"2026-07-18T06:33:55.122Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.123Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ab9027d93240c168e8ce4c05: done: 0.026178 sec","job_status":"done","queue_duration_s":0.29017,"scheduling_latency_s":0.290209,"redis_calls":7,"redis_duration_s":0.001511,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026178,"completed_at":"2026-07-18T06:33:55.440Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1929gitlab # [ 188.563101] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e87d9b9553c0af350f5e99bc","created_at":"2026-07-18T06:33:55.124Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.126Z","job_size_bytes":19,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e87d9b9553c0af350f5e99bc: start","job_status":"start","queue_duration_s":0.316173,"scheduling_latency_s":0.316212}1930gitlab # [ 188.563762] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.459Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e87d9b9553c0af350f5e99bc","created_at":"2026-07-18T06:33:55.124Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.126Z","job_size_bytes":19,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e87d9b9553c0af350f5e99bc: done: 0.017776 sec","job_status":"done","queue_duration_s":0.316173,"scheduling_latency_s":0.316212,"redis_calls":8,"redis_duration_s":0.001156,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017776,"completed_at":"2026-07-18T06:33:55.459Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1931gitlab # [ 188.606651] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.460Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2aba27810791db99570ddcac","created_at":"2026-07-18T06:33:55.126Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.127Z","job_size_bytes":66,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2aba27810791db99570ddcac: start","job_status":"start","queue_duration_s":0.333063,"scheduling_latency_s":0.333077}1932gitlab # [ 188.607663] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.471Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2aba27810791db99570ddcac","created_at":"2026-07-18T06:33:55.126Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.127Z","job_size_bytes":66,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2aba27810791db99570ddcac: done: 0.010866 sec","job_status":"done","queue_duration_s":0.333063,"scheduling_latency_s":0.333077,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010866,"completed_at":"2026-07-18T06:33:55.471Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1933gitlab # [ 188.650399] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.472Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"48a3b175a0f7e07925d326f3","created_at":"2026-07-18T06:33:55.128Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.129Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-48a3b175a0f7e07925d326f3: start","job_status":"start","queue_duration_s":0.342725,"scheduling_latency_s":0.342738}1934gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1935gitlab # [ 188.652744] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"48a3b175a0f7e07925d326f3","created_at":"2026-07-18T06:33:55.128Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.129Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-48a3b175a0f7e07925d326f3: done: 0.016682 sec","job_status":"done","queue_duration_s":0.342725,"scheduling_latency_s":0.342738,"redis_calls":7,"redis_duration_s":0.000911,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000679,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016682,"completed_at":"2026-07-18T06:33:55.489Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1936gitlab # [ 188.694118] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.489Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"e9b118f65ea601c56eaa7f59","created_at":"2026-07-18T06:33:55.167Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.170Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e9b118f65ea601c56eaa7f59: start","job_status":"start","queue_duration_s":0.319842,"scheduling_latency_s":0.319856}1937gitlab # [ 188.731836] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.507Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"e9b118f65ea601c56eaa7f59","created_at":"2026-07-18T06:33:55.167Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.170Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e9b118f65ea601c56eaa7f59: done: 0.017382 sec","job_status":"done","queue_duration_s":0.319842,"scheduling_latency_s":0.319856,"redis_calls":7,"redis_duration_s":0.00145,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017382,"completed_at":"2026-07-18T06:33:55.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 188.740412] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.510Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b8eb50e09ef4cfd3f87c2a27","created_at":"2026-07-18T06:33:55.170Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.172Z","job_size_bytes":6,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b8eb50e09ef4cfd3f87c2a27: start","job_status":"start","queue_duration_s":0.337773,"scheduling_latency_s":0.337812}1939gitlab # [ 188.755711] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.531Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b8eb50e09ef4cfd3f87c2a27","created_at":"2026-07-18T06:33:55.170Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.172Z","job_size_bytes":6,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b8eb50e09ef4cfd3f87c2a27: done: 0.020609 sec","job_status":"done","queue_duration_s":0.337773,"scheduling_latency_s":0.337812,"redis_calls":7,"redis_duration_s":0.001702,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020609,"completed_at":"2026-07-18T06:33:55.531Z","load_balancing_strategy":"primary","db_duration_s":0.000404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1940gitlab # [ 188.823756] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.532Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"73e51e058845943ad25ba2bb","created_at":"2026-07-18T06:33:55.173Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.176Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-73e51e058845943ad25ba2bb: start","job_status":"start","queue_duration_s":0.356389,"scheduling_latency_s":0.35642}1941gitlab # [ 188.824689] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.565Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"73e51e058845943ad25ba2bb","created_at":"2026-07-18T06:33:55.173Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.176Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-73e51e058845943ad25ba2bb: done: 0.032295 sec","job_status":"done","queue_duration_s":0.356389,"scheduling_latency_s":0.35642,"redis_calls":10,"redis_duration_s":0.001776,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000315124,"duration_s":0.032295,"completed_at":"2026-07-18T06:33:55.565Z","load_balancing_strategy":"primary","db_duration_s":0.00456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 188.878792] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.566Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"2e9e1bd14d05f4850d52762c","created_at":"2026-07-18T06:33:55.212Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.214Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2e9e1bd14d05f4850d52762c: start","job_status":"start","queue_duration_s":0.351254,"scheduling_latency_s":0.351268}1943gitlab # [ 188.879986] gitaly[1355]: time="2026-07-18T06:33:55.608Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:25.607" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.608" grpc.time_ms=0.798 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1944gitlab # [ 188.922535] gitaly[1355]: time="2026-07-18T06:33:55.900Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWXT6AEJVRMQPP86ASWSB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:25.199" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.899" grpc.time_ms=0.492 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.25 seconds)1946gitlab # [ 188.935122] gitaly[1355]: time="2026-07-18T06:33:55.912Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:25.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.912" grpc.time_ms=0.422 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947gitlab # [ 188.940057] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1341948gitlab # [ 188.969055] gitaly[1355]: time="2026-07-18T06:33:55.946Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWWV7QHHRY5Z6RYG22TZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:25.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:55.946" grpc.time_ms=0.445 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949gitlab # [ 188.988563] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.964Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"2e9e1bd14d05f4850d52762c","created_at":"2026-07-18T06:33:55.212Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-a1cbf2ec5031490e","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.214Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2e9e1bd14d05f4850d52762c: done: 0.39873 sec","job_status":"done","queue_duration_s":0.351254,"scheduling_latency_s":0.351268,"gitaly_calls":3,"gitaly_duration_s":0.300115,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.005005,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000604,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001326,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000176,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00016,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00038,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000605,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067688,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.373756441,"duration_s":0.39873,"completed_at":"2026-07-18T06:33:55.964Z","load_balancing_strategy":"primary","db_duration_s":0.031905,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1950gitlab # [ 188.995100] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.966Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"23dfe2b0fa949e028a481db2","created_at":"2026-07-18T06:33:55.600Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-34d3ca38d5e449b2","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.602Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-23dfe2b0fa949e028a481db2: start","job_status":"start","queue_duration_s":0.364429,"scheduling_latency_s":0.364445}1951gitlab # [ 189.034822] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:55.986Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"23dfe2b0fa949e028a481db2","created_at":"2026-07-18T06:33:55.600Z","trace_propagation_headers":{"sentry-trace":"2319d814e70f49619679a2220fed1df4-34d3ca38d5e449b2","baggage":"sentry-trace_id=2319d814e70f49619679a2220fed1df4,sentry-release=Unknown"},"correlation_id":"01KXSYWWV7QHHRY5Z6RYG22TZ4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:55.602Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-23dfe2b0fa949e028a481db2: done: 0.019985 sec","job_status":"done","queue_duration_s":0.364429,"scheduling_latency_s":0.364445,"redis_calls":7,"redis_duration_s":0.001439,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019985,"completed_at":"2026-07-18T06:33:55.986Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1952gitlab # [ 189.468158] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:33:56.445Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f52c7a5193695142d1ddddec","created_at":"2026-07-18T06:33:40.465Z","trace_propagation_headers":{"sentry-trace":"c2f1846027f14b40857cf9c84c7d8ab4-7669fad6226843e4","baggage":"sentry-trace_id=c2f1846027f14b40857cf9c84c7d8ab4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWENX0W90T94GZX7Z0ADH","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec43df877101bdaa602e26133163d28679fd667ab751704ab25424d50c00fbe2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:56.442Z","retry_count":0,"job_size_bytes":451,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f52c7a5193695142d1ddddec: start","job_status":"start","queue_duration_s":0.002648,"scheduling_latency_s":0.002697}1953gitlab # [ 189.519143] sidekiq-cluster[2051]: {"severity":"WARN","time":"2026-07-18T06:33:56.495Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"f52c7a5193695142d1ddddec","created_at":"2026-07-18T06:33:40.465Z","trace_propagation_headers":{"sentry-trace":"c2f1846027f14b40857cf9c84c7d8ab4-7669fad6226843e4","baggage":"sentry-trace_id=c2f1846027f14b40857cf9c84c7d8ab4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYWENX0W90T94GZX7Z0ADH","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:ec43df877101bdaa602e26133163d28679fd667ab751704ab25424d50c00fbe2","size_limiter":"validated","enqueued_at":"2026-07-18T06:33:56.442Z","retry_count":0,"job_size_bytes":451,"pid":2051,"sidekiq_tid":"m5tj","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-f52c7a5193695142d1ddddec: fail: 0.049163 sec","job_status":"fail","queue_duration_s":0.002648,"scheduling_latency_s":0.002697,"redis_calls":7,"redis_duration_s":0.002397,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049163,"completed_at":"2026-07-18T06:33:56.494Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.006271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1954gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1955gitlab # [ 190.042359] gitaly[1355]: time="2026-07-18T06:33:57.019Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYWYZ65XZV0VCSJS30Y2BK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:27.017" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:57.018" grpc.time_ms=0.796 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1956gitlab # [ 190.074826] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 841957gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.14 seconds)1958gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1959gitlab # [ 191.244207] gitaly[1355]: time="2026-07-18T06:33:58.221Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX02F8M0VNHAS6C0HZTCE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:28.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:58.219" grpc.time_ms=0.828 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 191.292758] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1731961gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.21 seconds)1962gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1963gitlab # [ 192.395150] gitaly[1355]: time="2026-07-18T06:33:59.371Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX18C29XNQ2WMZ5S2533M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:29.170" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:59.370" grpc.time_ms=0.823 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964gitlab # [ 192.430986] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:33:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 991965gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.14 seconds)1966gitlab # [ 192.879276] nix-daemon[3261]: accepted connection from pid 3512, user nixbot1967gitlab # [ 192.975615] gitaly[1355]: time="2026-07-18T06:33:59.953Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:29.351" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:33:59.951" grpc.time_ms=0.873 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 193.014915] gitaly[1355]: time="2026-07-18T06:33:59.992Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-18T06:34:09.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:33:59.981" grpc.time_ms=10.915 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969gitlab # [ 193.081087] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.057Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"dbf06f59fc813c0f44a4ce7e","created_at":"2026-07-18T06:34:00.051Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-06b8ff2eef164e34","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.055Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-dbf06f59fc813c0f44a4ce7e: start","job_status":"start","queue_duration_s":0.00199,"scheduling_latency_s":0.002042}1970gitlab # [ 193.130946] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:34:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2181971gitlab # [ 193.134267] nixbot[2971]: {"ts": "2026-07-18T06:34:00.109Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ce5370efa2035644cca25a7e5dba65a705872d9d \"HTTP/1.1 201 Created\""}1972gitlab # [ 193.254090] gitaly[1355]: time="2026-07-18T06:34:00.231Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:34:00.231" grpc.time_ms=0.386 pid=1355 span.kind=server system=grpc1973gitlab # [ 193.288199] gitaly[1355]: time="2026-07-18T06:34:00.265Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:34:00.265" grpc.time_ms=0.296 pid=1355 span.kind=server system=grpc1974gitlab # [ 193.316406] nix-daemon[3261]: accepted connection from pid 3537, user nixbot1975gitlab # [ 193.317127] gitaly[1355]: time="2026-07-18T06:34:00.293Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3902 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-18T06:34:00.292" grpc.time_ms=1.208 pid=1355 span.kind=server system=grpc1976gitlab # [ 193.327788] gitaly[1355]: time="2026-07-18T06:34:00.305Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:30.191" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:34:00.192" operations="[repack_refs]" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab # [ 193.329247] gitaly[1355]: time="2026-07-18T06:34:00.306Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:30.191" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:34:00.192" pid=1355 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-18 06:33:42.091595839 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root1978gitlab # [ 193.338311] gitaly[1355]: time="2026-07-18T06:34:00.315Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV diskcache=6fcca07a-4093-4007-99c2-e1eae465d132 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:30.191" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:34:00.192" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979gitlab # [ 193.338966] gitaly[1355]: time="2026-07-18T06:34:00.316Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=108 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7611 command.oublock=0 command.real_time_ms=115 command.spawn_token_fork_ms=0 command.system_time_ms=72 command.user_time_ms=36 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:30.191" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:34:00.192" grpc.time_ms=123.65 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 193.350768] gitaly[1355]: time="2026-07-18T06:34:00.328Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:30.191" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:34:00.192" pid=1355 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root1981gitlab # [ 193.351764] gitaly[1355]: time="2026-07-18T06:34:00.328Z" level=info msg="optimized repository" clean_stale_data_time=21.171964ms clean_work_trees_time="71.518µs" commit_graph_time="3.632µs" correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.47µs" pid=1355 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.19µs" storage_name=default system=housekeeping total_time=21.351317ms1982gitlab # [ 193.352642] gitaly[1355]: time="2026-07-18T06:34:00.328Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-18T06:34:30.191" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-18T06:34:00.192" pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses1984gitlab # [ 193.439688] gitaly[1355]: time="2026-07-18T06:34:00.415Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-18T06:34:30.414" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:34:00.415" grpc.time_ms=0.624 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 193.474943] gitaly[1355]: time="2026-07-18T06:34:00.452Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:30.449" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:34:00.452" grpc.time_ms=0.487 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 193.485990] gitaly[1355]: time="2026-07-18T06:34:00.463Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KXSYX2AMAJQMY55PG474S0HB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:34:00.455" grpc.time_ms=8.404 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc1987gitlab # [ 193.487218] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:34:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 591988gitlab # [ 193.508668] gitaly[1355]: time="2026-07-18T06:34:00.486Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX1TH8X9J6SJWSKYWQWFV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-18T06:34:30.485" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-18T06:34:00.485" grpc.time_ms=0.434 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989gitlab # [ 193.517298] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.493Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"dbf06f59fc813c0f44a4ce7e","created_at":"2026-07-18T06:34:00.051Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-06b8ff2eef164e34","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.055Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-dbf06f59fc813c0f44a4ce7e: done: 0.43638 sec","job_status":"done","queue_duration_s":0.00199,"scheduling_latency_s":0.002042,"gitaly_calls":4,"gitaly_duration_s":0.190983,"redis_calls":84,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.020832,"redis_read_bytes":174,"redis_write_bytes":21011,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001283,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.006671,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.002991,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13379,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005206,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000784,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00114,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147103,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.373546917,"duration_s":0.43638,"completed_at":"2026-07-18T06:34:00.493Z","load_balancing_strategy":"primary","db_duration_s":0.046459,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1990gitlab # [ 193.527310] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.494Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"272fc05afa6605a8b501a78f","created_at":"2026-07-18T06:34:00.181Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.184Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-272fc05afa6605a8b501a78f: start","job_status":"start","queue_duration_s":0.310163,"scheduling_latency_s":0.310179}1991gitlab # [ 193.571797] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.517Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"272fc05afa6605a8b501a78f","created_at":"2026-07-18T06:34:00.181Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.184Z","job_size_bytes":23,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-272fc05afa6605a8b501a78f: done: 0.022531 sec","job_status":"done","queue_duration_s":0.310163,"scheduling_latency_s":0.310179,"redis_calls":7,"redis_duration_s":0.001062,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022531,"completed_at":"2026-07-18T06:34:00.517Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00344,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1992gitlab # [ 193.617074] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.518Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4f0c6495bd5d0c40c8558c0d","created_at":"2026-07-18T06:34:00.375Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.377Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4f0c6495bd5d0c40c8558c0d: start","job_status":"start","queue_duration_s":0.140882,"scheduling_latency_s":0.140897}1993gitlab # [ 193.617591] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.535Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4f0c6495bd5d0c40c8558c0d","created_at":"2026-07-18T06:34:00.375Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.377Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4f0c6495bd5d0c40c8558c0d: done: 0.017043 sec","job_status":"done","queue_duration_s":0.140882,"scheduling_latency_s":0.140897,"redis_calls":7,"redis_duration_s":0.0009860000000000001,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017043,"completed_at":"2026-07-18T06:34:00.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1994gitlab # [ 193.656213] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4c7691dbca4e311566f91270","created_at":"2026-07-18T06:34:00.382Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.386Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4c7691dbca4e311566f91270: start","job_status":"start","queue_duration_s":0.149216,"scheduling_latency_s":0.149229}1995gitlab # [ 193.656884] gitlab-workhorse[1290]: time="2026-07-18T06:34:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671996gitlab # [ 193.657481] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:34:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 611997gitlab # [ 193.657947] gitaly[1355]: time="2026-07-18T06:34:00.526Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX2CJECDX482W9R7SKGRW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:34:00.519" pid=1355 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1998gitlab # [ 193.660202] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.576Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4c7691dbca4e311566f91270","created_at":"2026-07-18T06:34:00.382Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.386Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4c7691dbca4e311566f91270: done: 0.04086 sec","job_status":"done","queue_duration_s":0.149216,"scheduling_latency_s":0.149229,"redis_calls":7,"redis_duration_s":0.0009889999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04086,"completed_at":"2026-07-18T06:34:00.576Z","load_balancing_strategy":"primary","db_duration_s":0.027361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 193.711987] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.577Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"1f458fa579fe6c770ecb5596","created_at":"2026-07-18T06:34:00.398Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.400Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-1f458fa579fe6c770ecb5596: start","job_status":"start","queue_duration_s":0.177297,"scheduling_latency_s":0.177313}2000gitlab # [ 193.712693] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.592Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"1f458fa579fe6c770ecb5596","created_at":"2026-07-18T06:34:00.398Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.400Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-1f458fa579fe6c770ecb5596: done: 0.015014 sec","job_status":"done","queue_duration_s":0.177297,"scheduling_latency_s":0.177313,"redis_calls":10,"redis_duration_s":0.001281,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000348,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015014,"completed_at":"2026-07-18T06:34:00.592Z","load_balancing_strategy":"primary","db_duration_s":0.001983,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 193.768690] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.593Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c56dabef0c07590b3cc42b29","created_at":"2026-07-18T06:34:00.401Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.403Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c56dabef0c07590b3cc42b29: start","job_status":"start","queue_duration_s":0.190403,"scheduling_latency_s":0.190417}2002gitlab # [ 193.769548] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c56dabef0c07590b3cc42b29","created_at":"2026-07-18T06:34:00.401Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.403Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c56dabef0c07590b3cc42b29: done: 0.013139 sec","job_status":"done","queue_duration_s":0.190403,"scheduling_latency_s":0.190417,"redis_calls":7,"redis_duration_s":0.000928,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013139,"completed_at":"2026-07-18T06:34:00.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 193.833739] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"05cf57246377166e856f9693","created_at":"2026-07-18T06:34:00.404Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.406Z","job_size_bytes":19,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-05cf57246377166e856f9693: start","job_status":"start","queue_duration_s":0.201457,"scheduling_latency_s":0.20147}2004gitlab # [ 193.834606] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.620Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"05cf57246377166e856f9693","created_at":"2026-07-18T06:34:00.404Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.406Z","job_size_bytes":19,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-05cf57246377166e856f9693: done: 0.013031 sec","job_status":"done","queue_duration_s":0.201457,"scheduling_latency_s":0.20147,"redis_calls":8,"redis_duration_s":0.001023,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013031,"completed_at":"2026-07-18T06:34:00.620Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2005gitlab # [ 193.881660] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.621Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"99fe06af2e5ea00a0c351b4c","created_at":"2026-07-18T06:34:00.407Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.408Z","job_size_bytes":66,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-99fe06af2e5ea00a0c351b4c: start","job_status":"start","queue_duration_s":0.213095,"scheduling_latency_s":0.213135}2006gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses, in 0.48 seconds)2007(finished: subtest: eval and build post commit statuses, in 11.17 seconds)2008(finished: run the VM test script, in 240.82 seconds)2009gitlab # [ 193.882496] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.631Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"99fe06af2e5ea00a0c351b4c","created_at":"2026-07-18T06:34:00.407Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.408Z","job_size_bytes":66,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-99fe06af2e5ea00a0c351b4c: done: 0.009946 sec","job_status":"done","queue_duration_s":0.213095,"scheduling_latency_s":0.213135,"redis_calls":7,"redis_duration_s":0.0008489999999999999,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000619,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009946,"completed_at":"2026-07-18T06:34:00.631Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 193.937847] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.632Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b1e52a3b71685acf86b8b603","created_at":"2026-07-18T06:34:00.409Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.410Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b1e52a3b71685acf86b8b603: start","job_status":"start","queue_duration_s":0.222078,"scheduling_latency_s":0.222092}2011gitlab # [ 193.940338] gitaly[1355]: time="2026-07-18T06:34:00.526Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KXSYX2CJECDX482W9R7SKGRW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-18T06:34:00.519" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1355 remote_ip="2001:db8:1::1" span.kind=server system=grpc2012gitlab # [ 193.940874] gitlab-workhorse[1290]: gitlab 2001:db8:1::1 - - [2026/07/18:06:34:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ce5370efa2035644cca25a7e5dba65a705872d9d/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 4002013gitlab # [ 193.941253] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.658Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b1e52a3b71685acf86b8b603","created_at":"2026-07-18T06:34:00.409Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.410Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b1e52a3b71685acf86b8b603: done: 0.025524 sec","job_status":"done","queue_duration_s":0.222078,"scheduling_latency_s":0.222092,"redis_calls":7,"redis_duration_s":0.002401,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025524,"completed_at":"2026-07-18T06:34:00.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 194.009434] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.659Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"182fd5c016b3156312e4c806","created_at":"2026-07-18T06:34:00.461Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-18T06:34:00.463Z","job_size_bytes":3,"pid":2051,"sidekiq_tid":"m9pz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-182fd5c016b3156312e4c806: start","job_status":"start","queue_duration_s":0.195984,"scheduling_latency_s":0.196005}2015test script finished in 240.93s2016cleanup2017kill QemuMachine (pid 13)2018gitlab # [ 194.010185] sidekiq-cluster[2051]: {"severity":"INFO","time":"2026-07-18T06:34:00.673Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"182fd5c016b3156312e4c806","created_at":"2026-07-18T06:34:00.461Z","trace_propagation_headers":{"sentry-trace":"385549eb0f8e47f688f560a1b6d4aac6-cf6b11e5b2674404","baggage":"sentry-trace_id=385549eb0f8e47f688f560a1b6d4aac6,sentry-release=Unknown"},"correlation_id":"01KXSYX1TH8X9J6SJWSKYWQWFV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","workqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2019gitlab # e2020(finished: cleanup, in 1.28 seconds)2021additionally exposed symbols:2022 gitlab,2023 vlan1,2024 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2025[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-18T06:33:46.506Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2026[]2027[]2028[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-18T06:33:49.533724Z', 'started_at': None, 'finished_at': None}]2029[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2030[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2031[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2032[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2033[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2034[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': '2026-07-18T06:33:54.880Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-18T06:33:55.000Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2035[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': '2026-07-18T06:33:54.880Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-18T06:33:55.000Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2036[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': '2026-07-18T06:33:54.880Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-18T06:33:55.000Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2037[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': '2026-07-18T06:33:54.880Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-18T06:33:55.000Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2038[{'id': 2, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-18T06:33:49.954Z', 'started_at': None, 'finished_at': '2026-07-18T06:33:54.880Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'ce5370efa2035644cca25a7e5dba65a705872d9d', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-18T06:33:55.000Z', 'started_at': None, 'finished_at': '2026-07-18T06:34:00.042Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]