vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #381
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.Gxd11Vd8YD', fmt=raw size=107374182412(finished: start all VMs, in 49.46 seconds)13gitlab # mke2fs 1.47.4 (6-Mar-2025)14gitlab: waiting for unit setup-gitlab.service15gitlab # Discarding device blocks: 0/262144 done16gitlab: waiting for the VM to finish booting17gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes18gitlab # Filesystem UUID: a7f7d6dc-2fad-4f29-8986-54783656a56e19gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qaxw3gkzzf3ddlrqs5gkf4nfr9ql7da0-nixos-system-gitlab-test/init regInfo=/nix/store/s7rn85s9118iiay20fhhk55c0cvbnp6n-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 478059102 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000004] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000803] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000871] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002726] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002737] Using GB pages for direct mapping73gitlab # [ 0.002919] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002926] ACPI: Early table checksum verification disabled75gitlab # [ 0.002929] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002932] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002936] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002943] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002945] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002947] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002948] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002950] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002951] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002952] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002953] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002953] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002954] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002954] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003178] No NUMA configuration found90gitlab # [ 0.003179] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003182] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.005524] Zone ranges:93gitlab # [ 0.005526] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005527] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005529] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005530] Device empty97gitlab # [ 0.005530] Movable zone start for each node98gitlab # [ 0.005531] Early memory node ranges99gitlab # [ 0.005531] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005533] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005534] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005535] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005555] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005825] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.115823] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.116268] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.116284] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.116318] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.116321] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.116323] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.116324] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.116325] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.116326] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.116329] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.116330] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.116335] TSC deadline timer available117gitlab # [ 0.116339] CPU topo: Max. logical packages: 1118gitlab # [ 0.116340] CPU topo: Max. logical dies: 1119gitlab # [ 0.116341] CPU topo: Max. dies per package: 1120gitlab # [ 0.116344] CPU topo: Max. threads per core: 1121gitlab # [ 0.116345] CPU topo: Num. cores per package: 4122gitlab # [ 0.116345] CPU topo: Num. threads per package: 4123gitlab # [ 0.116346] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.116372] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.116389] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.116392] kvm-guest: setup PV sched yield127gitlab # [ 0.116411] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.116413] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.116414] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.116415] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.116417] Booting paravirtualized kernel on KVM132gitlab # [ 0.116421] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.120883] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.123137] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.123193] kvm-guest: PV spinlocks enabled136gitlab # [ 0.123195] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.123197] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qaxw3gkzzf3ddlrqs5gkf4nfr9ql7da0-nixos-system-gitlab-test/init regInfo=/nix/store/s7rn85s9118iiay20fhhk55c0cvbnp6n-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.123298] Unknown kernel command line parameters "regInfo=/nix/store/s7rn85s9118iiay20fhhk55c0cvbnp6n-closure-info/registration", will be passed to user space.139gitlab # [ 0.123311] random: crng init done140gitlab # [ 0.123311] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.132476] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.136825] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.136906] software IO TLB: area num 4.144gitlab # [ 0.209063] Fallback order for Node 0: 0145gitlab # [ 0.209072] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.209074] Policy zone: Normal147gitlab # [ 0.211765] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.222175] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.235096] allocated 12582912 bytes of page_ext150gitlab # [ 0.245248] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.245250] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.246075] Dynamic Preempt: lazy153gitlab # [ 0.246255] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.246256] rcu: RCU event tracing is enabled.155gitlab # [ 0.246256] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.246258] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.246258] Rude variant of Tasks RCU enabled.158gitlab # [ 0.246259] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.246260] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.246260] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.246291] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.246292] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.246293] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.250664] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.250956] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.250965] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.251077] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.254654] Console: colour VGA+ 80x25169gitlab # [ 0.254657] printk: legacy console [tty0] enabled170gitlab # [ 0.284925] printk: legacy console [ttyS0] enabled171gitlab # [ 0.392744] ACPI: Core revision 20250807172gitlab # [ 0.393663] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.395254] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.396299] x2apic enabled175gitlab # [ 0.397052] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.398021] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.399294] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.400792] kvm-guest: setup PV IPIs179gitlab # [ 0.402424] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.403473] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.405220] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.407305] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.408350] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.410217] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.411222] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.412218] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.413218] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.415038] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.415976] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.417026] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.418226] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.419218] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.420218] active return thunk: srso_alias_return_thunk194gitlab # [ 0.421239] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.422217] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.424033] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.425217] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.426216] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.427217] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.428217] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.429218] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.431014] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.432007] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.433010] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.434007] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.435004] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.465183] Freeing SMP alternatives memory: 44K208gitlab # [ 0.465821] pid_max: default: 32768 minimum: 301209gitlab # [ 0.466923] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.467322] landlock: Up and running.211gitlab # [ 0.468695] Yama: becoming mindful.212gitlab # [ 0.469426] LSM support for eBPF active213gitlab # [ 0.470418] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.471359] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.474630] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.475795] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.476226] ... version: 2218gitlab # [ 0.476965] ... bit width: 48219gitlab # [ 0.477220] ... generic counters: 6220gitlab # [ 0.477993] ... generic bitmap: 000000000000003f221gitlab # [ 0.478221] ... fixed-purpose counters: 0222gitlab # [ 0.478957] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.479219] ... value mask: 0000ffffffffffff224gitlab # [ 0.480164] ... max period: 00007fffffffffff225gitlab # [ 0.480960] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.481339] signal: max sigframe size: 3376227gitlab # [ 0.482163] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.482867] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.483407] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.488776] smp: Bringing up secondary CPUs ...231gitlab # [ 0.489603] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.490235] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.493318] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.494962] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.495766] Memory: 6011612K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.497220] devtmpfs: initialized237gitlab # [ 0.498034] x86/mm: Memory block size: 128MB238gitlab # [ 0.501275] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.502339] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.503325] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.504518] PM: RTC time: 09:27:34, date: 2026-09-14242gitlab # [ 0.508031] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.510330] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.511285] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.513285] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.514231] audit: initializing netlink subsys (disabled)247gitlab # [ 0.515273] audit: type=2000 audit(1789378054.253:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.515528] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.516221] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.517220] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.518219] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.519220] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.520233] cpuidle: using governor menu254gitlab # [ 0.522419] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.523439] PCI: Using configuration type 1 for base access256gitlab # [ 0.524221] PCI: Using configuration type 1 for extended access257gitlab # [ 0.525379] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.527535] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.529129] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.530125] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.531127] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.535097] ACPI: Added _OSI(Module Device)263gitlab # [ 0.537227] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.539227] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.546272] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.552468] ACPI: Interpreter enabled267gitlab # [ 0.553238] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.554015] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.555507] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.558235] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.560628] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.571578] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.573235] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.577100] acpiphp: Slot [3] registered275gitlab # [ 0.579251] acpiphp: Slot [4] registered276gitlab # [ 0.580263] acpiphp: Slot [5] registered277gitlab # [ 0.581336] acpiphp: Slot [6] registered278gitlab # [ 0.583267] acpiphp: Slot [7] registered279gitlab # [ 0.584263] acpiphp: Slot [8] registered280gitlab # [ 0.586263] acpiphp: Slot [9] registered281gitlab # [ 0.587280] acpiphp: Slot [10] registered282gitlab # [ 0.588265] acpiphp: Slot [11] registered283gitlab # [ 0.590263] acpiphp: Slot [12] registered284gitlab # [ 0.591310] acpiphp: Slot [13] registered285gitlab # [ 0.593272] acpiphp: Slot [14] registered286gitlab # [ 0.594263] acpiphp: Slot [15] registered287gitlab # [ 0.595241] acpiphp: Slot [16] registered288gitlab # [ 0.595984] acpiphp: Slot [17] registered289gitlab # [ 0.597244] acpiphp: Slot [18] registered290gitlab # [ 0.598027] acpiphp: Slot [19] registered291gitlab # [ 0.598239] acpiphp: Slot [20] registered292gitlab # [ 0.598967] acpiphp: Slot [21] registered293gitlab # [ 0.600240] acpiphp: Slot [22] registered294gitlab # [ 0.601000] acpiphp: Slot [23] registered295gitlab # [ 0.601239] acpiphp: Slot [24] registered296gitlab # [ 0.601972] acpiphp: Slot [25] registered297gitlab # [ 0.603245] acpiphp: Slot [26] registered298gitlab # [ 0.603992] acpiphp: Slot [27] registered299gitlab # [ 0.604248] acpiphp: Slot [28] registered300gitlab # [ 0.604967] acpiphp: Slot [29] registered301gitlab # [ 0.606241] acpiphp: Slot [30] registered302gitlab # [ 0.606978] acpiphp: Slot [31] registered303gitlab # [ 0.607289] PCI host bridge to bus 0000:00304gitlab # [ 0.608005] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.609140] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.610122] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.611220] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.613220] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.614231] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.615394] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.617726] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.620148] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.627300] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.629323] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.632232] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.635231] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.638231] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.641724] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.648327] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.653873] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.657964] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.661251] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.664814] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.683296] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.685236] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.686243] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.687550] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.690008] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.695256] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.696154] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.697008] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.709230] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.712869] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.716257] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.717220] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.718241] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.720532] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.725454] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.727898] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.728242] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.731454] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.734165] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.734986] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.736242] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.739479] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.743258] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.744161] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.746006] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.749145] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.752288] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.754240] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.755244] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.757660] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.763287] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.764132] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.764953] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.766154] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.769259] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.772252] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.773202] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.775216] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.780573] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.782893] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.783241] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.786489] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.789186] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.789864] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.791242] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.796976] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.798371] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.799381] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.800350] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.801300] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.804701] iommu: Default domain type: Translated375gitlab # [ 0.805079] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.806561] ACPI: bus type USB registered377gitlab # [ 0.807326] usbcore: registered new interface driver usbfs378gitlab # [ 0.808239] usbcore: registered new interface driver hub379gitlab # [ 0.809165] usbcore: registered new device driver usb380gitlab # [ 0.812306] NetLabel: Initializing381gitlab # [ 0.812933] NetLabel: domain hash size = 128382gitlab # [ 0.813220] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.814213] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.815149] PCI: Using ACPI for IRQ routing385gitlab # [ 0.816377] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.817215] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.817216] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.819223] vgaarb: loaded389gitlab # [ 0.819964] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.821220] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.825277] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.829744] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.830465] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.831983] pnp: PnP ACPI init395gitlab # [ 0.833168] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.844671] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.848315] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.850917] NET: Registered PF_INET protocol family399gitlab # [ 0.856086] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.891372] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.892880] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.894291] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.898148] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.899557] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.901042] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.902681] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.904127] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.905524] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.906560] NET: Registered PF_XDP protocol family410gitlab # [ 0.907472] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.908547] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.909618] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.910801] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.911960] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.913355] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.914410] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.916922] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.919437] PCI: CLS 0 bytes, default 64419gitlab # [ 0.920303] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.920672] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.921409] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.921531] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 0.945013] Initialise system trusted keyrings424gitlab # [ 0.946139] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.958069] Key type asymmetric registered426gitlab # [ 0.958813] Asymmetric key parser 'x509' registered427gitlab # [ 0.959705] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.961360] io scheduler mq-deadline registered429gitlab # [ 0.962187] io scheduler kyber registered430gitlab # [ 0.965864] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.967147] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.969519] Linux agpgart interface v0.103433gitlab # [ 0.970347] ACPI: bus type drm_connector registered434gitlab # [ 0.972906] usbcore: registered new interface driver usbserial_generic435gitlab # [ 0.974052] usbserial: USB Serial support registered for generic436gitlab # [ 0.975090] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 0.975090] Please enable it if your BIOS has the CPPC option.438gitlab # [ 0.977450] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 0.978880] drop_monitor: Initializing network drop monitor service440gitlab # [ 0.980041] NET: Registered PF_INET6 protocol family441gitlab # [ 0.981846] Segment Routing with IPv6442gitlab # [ 0.982537] In-situ OAM (IOAM) with IPv6443gitlab # [ 0.984728] IPI shorthand broadcast: enabled444gitlab # [ 0.989080] sched_clock: Marking stable (842014566, 146251639)->(1009587992, -21321787)445gitlab # [ 0.990842] registered taskstats version 1446gitlab # [ 0.991836] Loading compiled-in X.509 certificates447gitlab # [ 1.001319] Demotion targets for Node 0: null448gitlab # [ 1.002416] Key type .fscrypt registered449gitlab # [ 1.003128] Key type fscrypt-provisioning registered450gitlab # [ 1.004156] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.005118] ima: Allocated hash algorithm: sha1452gitlab # [ 1.005933] ima: No architecture policies found453gitlab # [ 1.006970] PM: Magic number: 10:948:474454gitlab # [ 1.008700] RAS: Correctable Errors collector initialized.455gitlab # [ 1.013153] clk: Disabling unused clocks456gitlab # [ 1.013865] PM: genpd: Disabling unused power domains457gitlab # [ 1.201604] Freeing initrd memory: 29392K458gitlab # [ 1.205196] Freeing unused decrypted memory: 2028K459gitlab # [ 1.208037] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.209181] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.211361] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.213089] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.274985] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.276174] Run /init as init process465gitlab # [ 1.292791] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.316008] fuse: init (API version 7.45)467gitlab # [ 1.324793] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.332711] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.335823] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.373182] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.709590] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.748535] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.750754] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.752938] systemd[1]: Running in initrd.475gitlab # [ 1.755508] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.758419] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.875585] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.917161] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.926910] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.928091] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.929552] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.930722] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.931837] systemd[1]: Reached target Path Units.484gitlab # [ 1.932720] systemd[1]: Reached target Slice Units.485gitlab # [ 1.933630] systemd[1]: Reached target Swaps.486gitlab # [ 1.934453] systemd[1]: Reached target Timer Units.487gitlab # [ 1.935476] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.936740] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.937895] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.938911] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.939938] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.940898] systemd[1]: Reached target Socket Units.493gitlab # [ 1.942821] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.944929] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 1.947036] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 1.951341] systemd[1]: Starting Journal Service...497gitlab # [ 1.955344] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 1.956358] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 1.961341] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 1.968172] systemd[1]: modprobe@configfs.service: Deactivated successfully.501gitlab # [ 1.971626] systemd[1]: Finished Load Kernel Module configfs.502gitlab # [ 1.975557] systemd[1]: Finished Create List of Static Device Nodes.503gitlab # [ 1.978430] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config504gitlab # [ 1.984593] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...505gitlab # [ 1.995794] systemd-journald[92]: Collecting audit messages is disabled.506gitlab # [ 2.004694] netfs: FS-Cache loaded507gitlab # [ 2.008846] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.508gitlab # [ 2.013508] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev509gitlab # [ 2.017505] 9pnet: Installing 9P2000 support510gitlab # [ 2.019743] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.511gitlab # [ 2.022188] systemd[1]: Finished Load Kernel Modules.512gitlab # [ 2.023462] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.513gitlab # [ 2.025233] systemd[1]: Finished Load Kernel Module 9pnet_virtio.514gitlab # [ 2.027755] systemd[1]: Starting Apply Kernel Variables...515gitlab # [ 2.029484] systemd[1]: Starting Create Static Device Nodes in /dev...516gitlab # [ 2.041974] systemd[1]: Finished Create Static Device Nodes in /dev.517gitlab # [ 2.043707] systemd[1]: Finished Apply Kernel Variables.518gitlab # [ 2.045030] systemd[1]: Reached target Preparation for Local File Systems.519gitlab # [ 2.046332] systemd[1]: Reached target Local File Systems.520gitlab # [ 1.900728] systemd-modules-load[94]: Using 4 probe threads521gitlab # [ 1.902342] systemd-modules-l[ 2.048431] systemd[1]: Starting Rule-based Manager for Device Events and Files...522gitlab # oad[94]: Inserted module 'virtio_balloon'523gitlab # [ 2.050006] systemd[1]: Started Journal Service.524gitlab # [ 1.904295] systemd-modules-load[94]: Inserted module 'dm_mod'525gitlab # [ 1.905559] systemd-modules-load[94]: Inserted module 'virtio_gpu'526gitlab # [ 1.907474] systemd[1]: Starting Create System Files and Directories...527gitlab # [ 1.920632] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 1.927036] systemd-udevd[104]: Using default interface naming scheme 'v261'.529gitlab # [ 1.937417] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 1.943096] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 1.944367] systemd[1]: Reached target System Initialization.532gitlab # [ 1.945174] systemd[1]: Reached target Basic System.533gitlab # [ 2.182101] virtio_blk virtio4: 4/0/0 default/read/poll queues534gitlab # [ 2.201198] SCSI subsystem initialized535gitlab # [ 2.203096] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)536gitlab # [ 2.207080] uhci_hcd 0000:00:01.2: UHCI Host Controller537gitlab # [ 2.208534] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1538gitlab # [ 2.209704] uhci_hcd 0000:00:01.2: detected 2 ports539gitlab # [ 2.217276] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140540gitlab # [ 2.219149] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18541gitlab # [ 2.220580] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1542gitlab # [ 2.221890] usb usb1: Product: UHCI Host Controller543gitlab # [ 2.222816] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd544gitlab # [ 2.223613] usb usb1: SerialNumber: 0000:00:01.2545gitlab # [ 2.229640] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12546gitlab # [ 2.229720] virtio_blk virtio5: 4/0/0 default/read/poll queues547gitlab # [ 2.232374] hub 1-0:1.0: USB hub found548gitlab # [ 2.233619] hub 1-0:1.0: 2 ports detected549gitlab # [ 2.233770] serio: i8042 KBD port at 0x60,0x64 irq 1550gitlab # [ 2.235142] serio: i8042 AUX port at 0x60,0x64 irq 12551gitlab # [ 2.239168] virtio_blk virtio5: [vdb] 11843776 512-byte logical blocks (6.06 GB/5.65 GiB)552gitlab # [ 2.241863] scsi host0: ata_piix553gitlab # [ 2.262180] scsi host1: ata_piix554gitlab # [ 2.262840] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0555gitlab # [ 2.265829] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0556gitlab # [ 2.122920] (udev-worker)[136]: Network interface NamePolicy= disabled on kernel command line.557gitlab # [ 2.124184] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.132048] (udev-worker)[122]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.559gitlab # [ 2.134354] (udev-worker)[122]: Network interface NamePolicy= disabled on kernel command line.560gitlab # [ 2.144225] systemd[1]: Found device /dev/disk/by-label/nix-store.561gitlab # [ 2.147250] systemd[1]: Found device /dev/disk/by-label/nixos.562gitlab # [ 2.148117] systemd[1]: Reached target Initrd Root Device.563gitlab # [ 2.148966] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...564gitlab # [ 2.150808] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones.565gitlab # [ 2.165848] systemd[1]: Finished Virtual Console Setup.566gitlab # [ 2.179552] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks567gitlab # [ 2.184376] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.568gitlab # [ 2.428767] ata2: found unknown device (class 0)569gitlab # [ 2.431512] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100570gitlab # [ 2.435604] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5571gitlab # [ 2.458718] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0572gitlab # [ 2.462295] usb 1-1: new full-speed USB device number 2 using uhci_hcd573gitlab # [ 2.489341] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray574gitlab # [ 2.515979] cdrom: Uniform CD-ROM driver Revision: 3.20575gitlab # [ 2.629862] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.631165] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.632298] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.632891] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.633480] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.663468] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.676879] usbcore: registered new interface driver usbhid582gitlab # [ 2.678940] usbhid: USB HID core driver583gitlab # [ 2.687989] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.691962] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.818505] systemd[1]: Mounting /sysroot...586gitlab # [ 3.105684] EXT4-fs (vda): mounted filesystem a7f7d6dc-2fad-4f29-8986-54783656a56e r/w with ordered data mode. Quota mode: none.587gitlab # [ 2.965438] systemd[1]: Mounted /sysroot.588gitlab # [ 2.967031] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 2.969300] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 2.971481] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 2.973605] systemd[1]: Mounting /sysroot/run...592gitlab # [ 2.975445] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 2.983115] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 2.986152] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 2.994094] systemd[1]: Mounted /sysroot/nix/.rw-store.596gitlab # [ 2.995888] systemd[1]: Mounted /sysroot/run.597gitlab # [ 2.997929] systemd-sysroot-fstab-check[180]: /sysroot should be mounted in the initrd, will request daemon-reload.598gitlab # [ 3.001251] systemd[1]: Starting rw-sysroot-nix-store.service...[ 3.149622] 9p: Installing v9fs 9p2000 file system support599gitlab # 600gitlab # [ 3.004993] systemd[1]: Reload requested from client PID 180 ('systemd-sysroot') (unit initrd-parse-etc.service)...601gitlab # [ 3.008914] s[ 3.155091] erofs (device vdb): mounted with root inode @ nid 128.602gitlab # ystemd[1]: Reloading...603gitlab # [ 3.062030] systemd[1]: Reloading finished in 63 ms.604gitlab # [ 3.083788] systemd-sysroot-fstab-check[180]: Requesting initrd-fs.target/start/replace...605gitlab # [ 3.088101] systemd[1]: Mounted /sysroot/nix/.ro-store.606gitlab # [ 3.089984] systemd[1]: Mounted /sysroot/tmp/shared.607gitlab # [ 3.091746] systemd[1]: Mounted /sysroot/tmp/xchg.608gitlab # [ 3.093443] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.095915] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.098457] systemd-sysroot-fstab-check[180]: Requesting swap.target/start/replace...611gitlab # [ 3.100998] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.103166] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.105496] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.107776] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.110804] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.113239] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.820538] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.872156] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.874557] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.876725] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.878766] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.904457] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.907360] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.926195] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.928445] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.930759] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 3.960827] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 3.963203] systemd[1]: Stopped target Basic System.629gitlab # [ 3.965733] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 3.967937] systemd[1]: Stopped target Path Units.631gitlab # [ 3.969934] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 3.974266] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 3.977155] systemd[1]: Stopped target Slice Units.634gitlab # [ 3.982525] systemd[1]: Stopped target Socket Units.635gitlab # [ 3.984715] systemd[1]: Stopped target System Initialization.636gitlab # [ 3.986761] systemd[1]: Stopped target Swaps.637gitlab # [ 3.988423] systemd[1]: Stopped target Timer Units.638gitlab # [ 3.990309] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 3.992408] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 3.994596] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 3.997277] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 3.999123] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.002294] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.004135] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.006164] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.007730] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.009895] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.011581] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.013918] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.016356] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.018828] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.020838] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.022506] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.024163] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.025472] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.026507] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.027800] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.029093] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.029999] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.031165] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.032230] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.033360] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.034306] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.035364] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.036498] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.037663] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.038514] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.039412] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.040588] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.041569] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.042723] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.043765] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.044848] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.068296] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.070484] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.071583] systemd[1]: Reached target Switch Root.677gitlab # [ 4.073421] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.198930] initrd-nixos-activation-start[232]: booting system configuration /nix/store/qaxw3gkzzf3ddlrqs5gkf4nfr9ql7da0-nixos-system-gitlab-test679gitlab # [ 4.242116] initrd-nixos-activation-start[232]: running activation script...680gitlab # [ 4.465644] initrd-nixos-activation-start[255]: setting up /etc...681gitlab # [ 4.601894] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.604548] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.606509] systemd[1]: Starting Switch Root...684gitlab # [ 4.624695] systemd[1]: Switching root.685gitlab # [ 4.832029] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.929053] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.292349] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.303400] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.305421] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.307611] systemd[1]: Detected first boot.691gitlab # [ 5.310453] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.468458] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.543414] systemd[1]: Applying preset policy.694gitlab # [ 5.674598] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.794883] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.798438] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.803597] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.809068] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.812693] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.816573] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.818922] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.822030] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.824827] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.826702] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.828599] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.830345] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.831927] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.833576] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.835381] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.837307] systemd[1]: Reached target Path Units.711gitlab # [ 5.838918] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.840577] systemd[1]: Reached target Slice Units.713gitlab # [ 5.842071] systemd[1]: Reached target Swaps.714gitlab # [ 5.845064] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.848425] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.849839] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.851450] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.852470] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.855019] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.856362] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.857980] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.859108] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.860304] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.861664] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.864429] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.865490] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.866444] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.869541] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.871669] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.874031] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.877330] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.887534] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.888769] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.895528] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.927780] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.931658] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.935639] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.941171] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.942993] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 5.954438] systemd[1]: Starting Journal Service...741gitlab # [ 5.969766] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 5.974432] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 5.982423] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 5.984501] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 5.985046] systemd-journald[327]: Collecting audit messages is enabled.746gitlab # [ 5.988397] systemd[1]: Starting Coldplug All udev Devices...747gitlab # [ 5.997137] systemd[1]: Listening on Journal Log Access Socket.748gitlab # [ 5.998226] systemd[1]: Mounted Huge Pages File System.749gitlab # [ 5.999362] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 6.002290] systemd[1]: Mounted Kernel Debug File System.751gitlab # [ 6.003314] systemd[1]: Mounted Kernel Trace File System.752gitlab # [ 6.004835] systemd[1]: Finished Create List of Static Device Nodes.753gitlab # [ 6.006115] systemd[1]: modprobe@configfs.service: Deactivated successfully.754gitlab # [ 5.860758] systemd[1]: Queued start job for default target Multi-User System.755gitlab # [ 5.862261] systemd[1]: Unnecessary job was removed for /dev/vdb.756gitlab # [ 6.009303] EXT4-fs (vda): re-mounted a7f7d6dc-2fad-4f29-8986-54783656a56e.757gitlab # [ 5.863453] systemd[1]: systemd-journald.service: Deactivated successfully.758gitlab # [ 6.010763] systemd[1]: Finished Load Kernel Module configfs.759gitlab # [ 5.865778] systemd-modules-load[328]: Using 3 probe threads760gitlab # [ 6.013577] systemd[1]: Started Journal Service.761gitlab # [ 5.869178] systemd[1]: Mounted FUSE Control File System.762gitlab # [ 5.872289] systemd-modules-load[328]: Inserted module 'tls'763gitlab # [ 5.873199] systemd[1]: Finished Remount Root and Kernel File Systems.764gitlab # [ 6.021504] loop: module loaded765gitlab # [ 5.875914] systemd-modules-load[328]: Inserted module 'loop'766gitlab # [ 5.878778] systemd[1]: Finished Load Kernel Modules.767gitlab # [ 5.880106] systemd[1]: Listening on Disk Image Download Service Socket.768gitlab # [ 5.883148] systemd[1]: Mounting Kernel Configuration File System...769gitlab # [ 5.884421] systemd[1]: Starting Firewall...770gitlab # [ 5.888057] systemd-oomd[330]: No swap; memory pressure usage will be degraded771gitlab # [ 5.889460] systemd[1]: Starting Flush Journal to Persistent Storage...772gitlab # [ 5.893164] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore773gitlab # [ 5.895031] systemd[1]: Starting Load/Save OS Random Seed...774gitlab # [ 5.898142] systemd[1]: Starting Apply Kernel Variables...775gitlab # [ 6.045377] systemd-journald[327]: Received client request to flush runtime journal.776gitlab # [ 5.936747] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...777gitlab # [ 5.938149] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os778gitlab # [ 5.941138] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.779gitlab # [ 5.942389] systemd[1]: Mounted Kernel Configuration File System.780gitlab # [ 5.943454] systemd[1]: Finished Apply Kernel Variables.781gitlab # [ 5.944258] systemd[1]: Finished Load/Save OS Random Seed.782gitlab # [ 5.945516] systemd[1]: Reached target First Boot Complete.783gitlab # [ 5.946333] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.784gitlab # [ 5.947639] systemd[1]: Starting Create Static Device Nodes in /dev...785gitlab # [ 5.948603] systemd[1]: Finished Flush Journal to Persistent Storage.786gitlab # [ 5.949526] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 5.950420] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 5.951351] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 5.970525] systemd-udevd[364]: Using default interface naming scheme 'v261'.790gitlab # [ 5.998599] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.001085] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.072238] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.095750] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.120071] (udev-worker)[417]: Network interface NamePolicy= disabled on kernel command line.795gitlab # [ 6.135971] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.796gitlab # [ 6.145584] (udev-worker)[440]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 6.147839] (udev-worker)[440]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.324700] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3799gitlab # [ 6.345604] ACPI: button: Power Button [PWRF]800gitlab # [ 6.353757] rtc_cmos 00:05: RTC can wake from S4801gitlab # [ 6.208702] systemd[1]: Condition check resulted in Virtio network device being skipped.802gitlab # [ 6.362927] rtc_cmos 00:05: registered as rtc0803gitlab # [ 6.362976] parport_pc 00:03: reported by Plug and Play ACPI804gitlab # [ 6.363198] rtc_cmos 00:05: setting system clock to 2026-09-14T09:27:39 UTC (1789378059)805gitlab # [ 6.363471] parport0: PC-style at 0x378806gitlab # [ 6.363866] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs807gitlab # [ 6.364255] , irq 7 [PCSPP(,...)]808gitlab # [ 6.374989] systemd-journald[327]: Time jumped backwards, rotating.809gitlab # [ 6.378656] Floppy drive(s): fd0 is 2.88M AMI BIOS810gitlab # [ 6.391125] FDC 0 is a S82078B811gitlab # [ 6.395560] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console812gitlab # [ 6.254162] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore813gitlab # [ 6.260952] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67814gitlab # [ 6.263792] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore815gitlab # [ 6.265872] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os816gitlab # [ 6.267455] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os817gitlab # [ 6.269048] systemd[1]: Finished Firewall.818gitlab # [ 6.396734] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0819gitlab # [ 6.396793] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD820gitlab # [ 6.417017] Console: switching to colour dummy device 80x25821gitlab # [ 6.419870] [drm] Found bochs VGA, ID 0xb0c5.822gitlab # [ 6.419872] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.823gitlab # [ 6.429032] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic824gitlab # [ 6.429817] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0825gitlab # [ 6.430589] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5826gitlab # [ 6.432485] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4827gitlab # [ 6.447907] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6828gitlab # [ 6.449182] Console: switching to colour frame buffer device 160x50829gitlab # [ 6.452126] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device830gitlab # [ 6.497372] mousedev: PS/2 mouse device common for all mice831gitlab # [ 6.377464] systemd[1]: Starting Virtual Console Setup...832gitlab # [ 6.395200] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.397045] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.410152] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 6.572084] ppdev: user-space parallel port driver836gitlab # [ 6.427161] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.837gitlab # [ 6.431715] systemd[1]: Stopped Virtual Console Setup.838gitlab # [ 6.436678] systemd[1]: Starting Virtual Console Setup...839gitlab # [ 6.603813] kvm_amd: TSC scaling supported840gitlab # [ 6.604476] kvm_amd: Nested Virtualization enabled841gitlab # [ 6.605088] kvm_amd: Nested Paging enabled842gitlab # [ 6.605758] kvm_amd: LBR virtualization supported843gitlab # [ 6.606357] kvm_amd: Virtual VMLOAD VMSAVE supported844gitlab # [ 6.606955] kvm_amd: Virtual GIF supported845gitlab # [ 6.607546] kvm_amd: Virtual NMI enabled846gitlab # [ 6.643582] EDAC MC: Ver: 3.0.0847gitlab # [ 6.504520] systemd-vconsole-setup[501]: Configuration of first virtual console was skipped, ignoring remaining ones.848gitlab # [ 6.506196] systemd[1]: Finished Virtual Console Setup.849gitlab # [ 6.652825] systemd[1]: Mounting /run/wrappers...850gitlab # [ 6.679415] systemd[1]: Mounted /run/wrappers.851gitlab # [ 6.681502] systemd[1]: Reached target Local File Systems.852gitlab # [ 6.683451] systemd[1]: Listening on Boot Loader Control Service Socket.853gitlab # [ 6.686056] systemd[1]: Starting register-nix-paths.service...854gitlab # [ 6.688512] systemd[1]: Starting Create SUID/SGID Wrappers...855gitlab # [ 6.690517] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.856gitlab # [ 6.693506] systemd[1]: Starting Save Transient machine-id to Disk...857gitlab # [ 6.699215] systemd[1]: Starting Create System Files and Directories...858gitlab # [ 6.733252] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.859gitlab # [ 6.736528] systemd[1]: Finished Save Transient machine-id to Disk.860gitlab # [ 6.742560] systemd[1]: Finished Create System Files and Directories.861gitlab # [ 6.745881] systemd[1]: Starting Rebuild Journal Catalog...862gitlab # [ 6.747875] systemd[1]: Starting Record System Boot/Shutdown in UTMP...863gitlab # [ 6.771820] systemd[1]: Finished Record System Boot/Shutdown in UTMP.864gitlab # [ 6.785511] systemd[1]: Finished Rebuild Journal Catalog.865gitlab # [ 6.787086] systemd[1]: Starting Update is Completed...866gitlab # [ 6.802546] systemd[1]: Finished Update is Completed.867gitlab # [ 6.847412] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.868gitlab # [ 6.848588] systemd[1]: Finished Create SUID/SGID Wrappers.869gitlab # [ 7.103076] systemd[1]: Finished register-nix-paths.service.870gitlab # [ 7.105531] systemd[1]: Reached target System Initialization.871gitlab # [ 7.107795] systemd[1]: Started Discard unused filesystem blocks once a week.872gitlab # [ 7.110466] systemd[1]: Started logrotate.timer.873gitlab # [ 7.112436] systemd[1]: Started Daily Cleanup of Temporary Directories.874gitlab # [ 7.114811] systemd[1]: Reached target Timer Units.875gitlab # [ 7.116759] systemd[1]: Listening on D-Bus System Message Bus Socket.876gitlab # [ 7.119126] systemd[1]: Listening on Nix Daemon Socket.877gitlab # [ 7.121207] systemd[1]: Starting nixbot CI service socket...878gitlab # [ 7.123424] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.879gitlab # [ 7.126521] systemd[1]: Starting D-Bus System Message Bus...880gitlab # [ 7.128694] systemd[1]: Listening on nixbot CI service socket.881gitlab # [ 7.130857] systemd[1]: Reached target Socket Units.882gitlab # [ 7.183711] dbus-broker-launch[584]: Looking up NSS user entry for 'systemd-timesync'...883gitlab # [ 7.187159] dbus-broker-launch[584]: NSS returned no entry for 'systemd-timesync'884gitlab # [ 7.190067] dbus-broker-launch[584]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"885gitlab # [ 7.195369] systemd[1]: Started D-Bus System Message Bus.886gitlab # [ 7.213880] dbus-broker-launch[584]: Ready887gitlab # [ 7.218247] systemd[1]: Reached target Basic System.888gitlab # [ 7.221254] systemd[1]: Started backdoor.service.889gitlab # [ 7.226139] systemd[1]: Starting gitlab-config.service...890gitlab # [ 7.228753] systemd[1]: Starting Import lastlog data into lastlog2 database...891gitlab # [ 7.236833] systemd[1]: Starting Logrotate configuration check...892gitlab # [ 7.241625] systemd[1]: Starting Name Service Cache Daemon (nsncd)...893gitlab # [ 7.244329] systemd[1]: Starting Post-Boot Actions...894gitlab # [ 7.250699] systemd[1]: Started Reset console on configuration changes.895gitlab # [ 7.261311] systemd[1]: Starting resolvconf update...896gitlab # connecting to host...897gitlab # [ 7.273274] logrotate[589]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.898gitlab # [ 7.273716] logrotate[589]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf899gitlab # [ 7.274394] logrotate[589]: note: 'monthly' overrides previously specified 'weekly'900gitlab # [ 7.275102] logrotate[589]: Reading state from file: /var/lib/logrotate.status901gitlab # [ 7.276066] logrotate[589]: state file /var/lib/logrotate.status does not exist902gitlab # [ 7.277646] logrotate[589]: Allocating hash table for state file, size 64 entries903gitlab # [ 7.278927] logrotate[589]: Handling 4 logs904gitlab: Guest shell says: b'Spawning backdoor root shell...\n'905gitlab # [ 7.279935] logrotate[589]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed906gitlab # [ 7.280926] logrotate[589]: considering log /var/log/btmp907gitlab: connected to guest root shell908gitlab # [ 7.281548] logrotate[589]: Creating new state909gitlab: (connecting took 7.92 seconds)910gitlab # [ 7.282754] logrotate[589]: Now: 2026-09-14 09:27911gitlab: (finished: waiting for the VM to finish booting, in 7.92 seconds)912gitlab # [ 7.283900] logrotate[589]: Last rotated at 2026-09-14 09:00913gitlab # [ 7.284847] logrotate[589]: log does not need rotating (log has already been rotated)914gitlab # [ 7.287955] logrotate[589]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed915gitlab # [ 7.288979] logrotate[589]: considering log /var/log/wtmp916gitlab # [ 7.290190] logrotate[589]: Creating new state917gitlab # [ 7.290911] logrotate[589]: Now: 2026-09-14 09:27918gitlab # [ 7.292029] logrotate[589]: Last rotated at 2026-09-14 09:00919gitlab # [ 7.292794] logrotate[589]: log does not need rotating (log has already been rotated)920gitlab # [ 7.293897] logrotate[589]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed921gitlab # [ 7.294656] logrotate[589]: switching euid from 0 to 165 and egid from 0 to 165 (pid 589)922gitlab # [ 7.296050] logrotate[589]: considering log /var/gitlab/state/log/*.log923gitlab # [ 7.296507] logrotate[589]: log /var/gitlab/state/log/*.log does not exist -- skipping924gitlab # [ 7.297146] logrotate[589]: Creating new state925gitlab # [ 7.297688] logrotate[589]: switching euid from 165 to 0 and egid from 165 to 0 (pid 589)926gitlab # [ 7.299500] logrotate[589]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed927gitlab # [ 7.300111] logrotate[589]: switching euid from 0 to 60 and egid from 0 to 60 (pid 589)928gitlab # [ 7.300537] logrotate[589]: considering log /var/log/nginx/*.log929gitlab # [ 7.301266] logrotate[589]: log /var/log/nginx/*.log does not exist -- skipping930gitlab # [ 7.301682] logrotate[589]: Creating new state931gitlab # [ 7.302190] logrotate[589]: not running postrotate script, since no logs were rotated932gitlab # [ 7.302637] logrotate[589]: switching euid from 60 to 0 and egid from 60 to 0 (pid 589)933gitlab # [ 7.303229] systemd[1]: Finished Logrotate configuration check.934gitlab # [ 7.303691] systemd[1]: Finished Post-Boot Actions.935gitlab # [ 7.328116] systemd[1]: Finished Import lastlog data into lastlog2 database.936gitlab # [ 7.333595] nsncd[590]: Sep 14 09:27:40.616 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"937gitlab # [ 7.334892] systemd[1]: Started Name Service Cache Daemon (nsncd).938gitlab # [ 7.335607] systemd[1]: Reached target Host and Network Name Lookups.939gitlab # [ 7.336096] systemd[1]: Reached target User and Group Name Lookups.940gitlab # [ 7.339240] systemd[1]: Starting User Login Management...941gitlab # [ 7.354679] systemd[1]: Stopped target Host and Network Name Lookups.942gitlab # [ 7.355151] systemd[1]: Stopping Host and Network Name Lookups...943gitlab # [ 7.355635] systemd[1]: Stopped target User and Group Name Lookups.944gitlab # [ 7.356160] systemd[1]: Stopping User and Group Name Lookups...945gitlab # [ 7.356657] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...946gitlab # [ 7.360549] systemd[1]: nscd.service: Deactivated successfully.947gitlab # [ 7.361705] systemd[1]: Stopped Name Service Cache Daemon (nsncd).948gitlab # [ 7.362033] systemd[1]: nscd.service: Consumed 19ms CPU time over 120ms wall clock time, 7.3M memory peak, 5.5M read from disk.949gitlab # [ 7.365163] systemd[1]: Starting Name Service Cache Daemon (nsncd)...950gitlab # [ 7.384805] systemd[1]: Started Name Service Cache Daemon (nsncd).951gitlab # [ 7.385367] systemd[1]: Reached target Host and Network Name Lookups.952gitlab # [ 7.386431] systemd[1]: Reached target User and Group Name Lookups.953gitlab # [ 7.387490] nsncd[681]: Sep 14 09:27:40.670 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"954gitlab # [ 7.394675] systemd-logind[652]: New seat seat0.955gitlab # [ 7.398201] systemd[1]: Finished resolvconf update.956gitlab # [ 7.399029] systemd-logind[652]: Watching system buttons on /dev/input/event2 (Power Button)957gitlab # [ 7.400212] systemd-logind[652]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)958gitlab # [ 7.401507] systemd-logind[652]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)959gitlab # [ 7.402778] systemd[1]: Started User Login Management.960gitlab # [ 7.403592] systemd[1]: Reached target Preparation for Network.961gitlab # [ 7.404508] systemd[1]: Starting DHCP Client...962gitlab # [ 7.405198] systemd[1]: Starting linger-users.service...963gitlab # [ 7.405957] systemd[1]: Starting Address configuration of eth1...964gitlab # [ 7.410214] systemd[1]: Starting Extra networking commands....965gitlab # [ 7.423725] systemd[1]: linger-users.service: Deactivated successfully.966gitlab # [ 7.425640] systemd[1]: Finished linger-users.service.967gitlab # [ 7.434633] network-addresses-eth1-start[705]: adding address 192.168.1.1/24... done968gitlab # [ 7.437732] network-addresses-eth1-start[705]: adding address 2001:db8:1::1/64... done969gitlab # [ 7.445934] systemd[1]: Finished Address configuration of eth1.970gitlab # [ 7.473119] dhcpcd[715]: dhcpcd-10.3.2 starting971gitlab # [ 7.474720] systemd[1]: Finished Extra networking commands..972gitlab # [ 7.476774] systemd[1]: Reached target Network.973gitlab # [ 7.477655] dhcpcd[771]: dev: loaded udev974gitlab # [ 7.478347] systemd[1]: Starting gitlab-workhorse.service...975gitlab # [ 7.479148] systemd[1]: Starting Nginx Web Server...976gitlab # [ 7.480832] systemd[1]: Starting PostgreSQL Server...977gitlab # [ 7.484449] systemd[1]: Starting Redis Server - redis-gitlab...978gitlab # [ 7.633187] 8021q: 802.1Q VLAN Support v1.8979gitlab # [ 7.633738] 8021q: adding VLAN 0 to HW filter on device eth1980gitlab # [ 7.488608] systemd[1]: Starting Permit User Sessions...981gitlab # [ 7.499908] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.982gitlab # [ 7.510570] systemd[1]: Finished Permit User Sessions.983gitlab # [ 7.512669] systemd[1]: Started Getty on tty1.984gitlab # [ 7.513126] systemd[1]: Reached target Login Prompts.985gitlab # [ 7.717219] cfg80211: Loading compiled-in X.509 certificates for regulatory database986gitlab # [ 7.722002] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'987gitlab # [ 7.722851] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'988gitlab # [ 7.724318] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2989gitlab # [ 7.725373] cfg80211: failed to load regulatory.db990gitlab # [ 7.751070] 8021q: adding VLAN 0 to HW filter on device eth0991gitlab # [ 7.606357] dhcpcd[771]: eth0: waiting for carrier992gitlab # [ 7.606748] dhcpcd[771]: eth0: carrier acquired993gitlab # [ 7.608907] dhcpcd[771]: DUID 00:01:00:01:32:3a:7a:8c:52:54:00:12:34:56994gitlab # [ 7.609526] dhcpcd[771]: eth0: IAID 00:12:34:56995gitlab # [ 7.610103] dhcpcd[771]: eth0: adding address fe80::5054:ff:fe12:3456996gitlab # [ 7.628562] nginx-pre-start[800]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok997gitlab # [ 7.630095] nginx-pre-start[800]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful998gitlab # [ 7.630667] redis-server[796]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.999gitlab # [ 7.631278] redis-server[796]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1000gitlab # [ 7.631868] redis-server[796]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=796, just started1001gitlab # [ 7.632528] redis-server[796]: Configuration loaded1002gitlab # [ 7.633022] redis[796]: monotonic clock: POSIX clock_gettime1003gitlab # [ 7.633510] redis[796]: Running mode=standalone, port=0.1004gitlab # [ 7.634094] redis[796]: Server initialized1005gitlab # [ 7.634649] redis[796]: Ready to accept connections unix1006gitlab # [ 7.635354] systemd[1]: Started Redis Server - redis-gitlab.1007gitlab # [ 7.635778] redis[796]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1008gitlab # [ 7.642257] postgresql-pre-start[798]: The files belonging to this database system will be owned by user "postgres".1009gitlab # [ 7.642559] postgresql-pre-start[798]: This user must also own the server process.1010gitlab # [ 7.646073] systemd[1]: Started Nginx Web Server.1011gitlab # [ 7.649796] postgresql-pre-start[798]: The database cluster will be initialized with locale "en_US.UTF-8".1012gitlab # [ 7.650173] postgresql-pre-start[798]: The default database encoding has accordingly been set to "UTF8".1013gitlab # [ 7.650640] postgresql-pre-start[798]: The default text search configuration will be set to "english".1014gitlab # [ 7.651160] postgresql-pre-start[798]: Data page checksums are enabled.1015gitlab # [ 7.651802] postgresql-pre-start[798]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1016gitlab # [ 7.655892] postgresql-pre-start[798]: creating subdirectories ... ok1017gitlab # [ 7.656201] postgresql-pre-start[798]: selecting dynamic shared memory implementation ... posix1018gitlab # [ 7.690727] postgresql-pre-start[798]: selecting default "max_connections" ... 1001019gitlab # [ 7.734619] postgresql-pre-start[798]: selecting default "shared_buffers" ... 128MB1020gitlab # [ 7.973268] systemd[1]: Started gitlab-workhorse.service.1021gitlab # [ 8.090398] postgresql-pre-start[798]: selecting default time zone ... UTC1022gitlab # [ 8.093341] postgresql-pre-start[798]: creating configuration files ... ok1023gitlab # [ 8.185431] gitlab-workhorse[832]: time="2026-09-14T09:27:41Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41024gitlab # [ 8.187241] gitlab-workhorse[832]: time="2026-09-14T09:27:41Z" level=info msg="Using redis/go-redis"1025gitlab # [ 8.188616] gitlab-workhorse[832]: time="2026-09-14T09:27:41Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1026gitlab # [ 8.257369] postgresql-pre-start[798]: running bootstrap script ... ok1027gitlab # [ 8.697392] postgresql-pre-start[798]: performing post-bootstrap initialization ... ok1028gitlab # [ 8.937089] postgresql-pre-start[798]: syncing data to disk ... ok1029gitlab # [ 8.939409] postgresql-pre-start[798]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 8.942457] postgresql-pre-start[798]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 8.947423] postgresql-pre-start[798]: Success. You can now start the database server using:1032gitlab # [ 8.950236] postgresql-pre-start[798]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 9.031213] dhcpcd[771]: eth0: soliciting a DHCP lease1034gitlab # [ 9.187406] NET: Registered PF_PACKET protocol family1035gitlab # [ 9.048858] dhcpcd[771]: eth0: offered 10.0.2.15 from 10.0.2.21036gitlab # [ 9.056157] dhcpcd[771]: eth0: probing address 10.0.2.15/241037gitlab # [ 9.079024] postgres[856]: [856] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1038gitlab # [ 9.081564] postgres[856]: [856] LOG: listening on IPv6 address "::1", port 54321039gitlab # [ 9.082683] postgres[856]: [856] LOG: listening on IPv4 address "127.0.0.1", port 54321040gitlab # [ 9.087477] postgres[856]: [856] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1041gitlab # [ 9.105091] postgres[868]: [868] LOG: database system was shut down at 2026-09-14 09:27:41 GMT1042gitlab # [ 9.115390] postgres[856]: [856] LOG: database system is ready to accept connections1043gitlab # [ 9.120927] systemd[1]: Started PostgreSQL Server.1044gitlab # [ 9.125537] systemd[1]: Starting PostgreSQL Setup Scripts...1045gitlab # [ 9.281762] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[876]: mkdir -p /var/gitlab/state/home/.ssh: OK1046gitlab # [ 9.282673] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[876]: chmod 700 /var/gitlab/state/home/.ssh: OK1047gitlab # [ 9.302038] postgresql-setup-start[884]: CREATE DATABASE1048gitlab # [ 9.328230] postgresql-setup-start[899]: CREATE ROLE1049gitlab # [ 9.333451] systemd[1]: Finished gitlab-config.service.1050gitlab # [ 9.336143] systemd[1]: Started gitaly.service.1051gitlab # [ 9.340352] postgresql-setup-start[902]: ALTER DATABASE1052gitlab # [ 9.363508] postgresql-setup-start[910]: CREATE ROLE1053gitlab # [ 9.365672] systemd[1]: Finished PostgreSQL Setup Scripts.1054gitlab # [ 9.366296] systemd[1]: Reached target PostgreSQL.1055gitlab # [ 9.367826] systemd[1]: Starting gitlab-db-config.service...1056gitlab # [ 9.368886] systemd[1]: Starting gitlab-postgresql.service...1057gitlab # [ 9.462440] gitlab-postgresql-start[924]: CREATE DATABASE1058gitlab # [ 9.507672] gitlab-postgresql-start[934]: CREATE EXTENSION1059gitlab # [ 9.543239] dhcpcd[771]: eth0: soliciting an IPv6 router1060gitlab # [ 9.544718] dhcpcd[771]: eth0: Router Advertisement from fe80::21061gitlab # [ 9.545571] dhcpcd[771]: eth0: adding address fec0::5054:ff:fe12:3456/641062gitlab # [ 9.546834] dhcpcd[771]: eth0: adding route to fec0::/641063gitlab # [ 9.547768] dhcpcd[771]: eth0: adding default route via fe80::21064gitlab # [ 9.590247] gitlab-postgresql-start[942]: CREATE EXTENSION1065gitlab # [ 9.594631] systemd[1]: Finished gitlab-postgresql.service.1066gitlab # [ 9.778233] gitaly[905]: time="2026-09-14T09:27:43.060Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9051067gitlab # [ 9.788508] gitaly[905]: time="2026-09-14T09:27:43.061Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9051068gitlab # [ 9.791730] gitaly[905]: time="2026-09-14T09:27:43.061Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9051069gitlab # [ 9.795899] gitaly[905]: time="2026-09-14T09:27:43.061Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9051070gitlab # [ 9.799309] gitaly[905]: time="2026-09-14T09:27:43.061Z" level=info msg="Starting Gitaly" pid=905 version=1071gitlab # [ 9.955594] gitaly[905]: time="2026-09-14T09:27:43.238Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=175 pid=9051072gitlab # [ 9.957651] gitaly[905]: time="2026-09-14T09:27:43.238Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9051073gitlab # [ 9.959513] gitaly[905]: time="2026-09-14T09:27:43.238Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9051074gitlab # [ 9.961469] gitaly[905]: time="2026-09-14T09:27:43.238Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-905/git-exec-2856507490.d/git pid=9051075gitlab # [ 9.967851] gitaly[905]: time="2026-09-14T09:27:43.250Z" level=info msg="finished detecting git version" duration_ms=11 pid=9051076gitlab # [ 9.969822] gitaly[905]: time="2026-09-14T09:27:43.250Z" level=info msg="using Git version" pid=905 version=2.53.g1077gitlab # [ 9.971448] gitaly[905]: time="2026-09-14T09:27:43.251Z" level=info msg="clearing disk cache object folder" pid=905 storage=default1078gitlab # [ 9.973348] gitaly[905]: time="2026-09-14T09:27:43.252Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2018243856 pid=905 storage=default1079gitlab # [ 9.976296] gitaly[905]: time="2026-09-14T09:27:43.252Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2018243856 pid=905 storage=default1080gitlab # [ 9.979502] gitaly[905]: time="2026-09-14T09:27:43.252Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9051081gitlab # [ 9.982445] gitaly[905]: time="2026-09-14T09:27:43.252Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9051082gitlab # [ 9.985428] gitaly[905]: time="2026-09-14T09:27:43.252Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2018243856 clear_duration_ms=0 pid=905 storage=default1083gitlab # [ 10.008909] gitaly[905]: time="2026-09-14T09:27:43.291Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9051084gitlab # [ 10.011580] gitaly[905]: time="2026-09-14T09:27:43.291Z" level=info msg="counting repositories" pid=9051085gitlab # [ 10.011884] gitaly[905]: time="2026-09-14T09:27:43.293Z" level=info msg="starting to count repositories" pid=905 storage_path=/var/gitlab/state/repositories1086gitlab # [ 10.012539] gitaly[905]: time="2026-09-14T09:27:43.291Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=905 schema=unix1087gitlab # [ 10.013087] gitaly[905]: time="2026-09-14T09:27:43.293Z" level=info msg="listening at address" address=/run/gitaly/gitaly-905/sock.d/intern pid=905 schema=unix1088gitlab # [ 10.013558] gitaly[905]: time="2026-09-14T09:27:43.293Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=905 storage_path=/var/gitlab/state/repositories1089gitlab # [ 10.013743] gitaly[905]: time="2026-09-14T09:27:43.293Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9051090gitlab # [ 10.014972] gitaly[905]: time="2026-09-14T09:27:43.292Z" level=info msg="finished tempdir cleaner walk" pid=905 storage=default time_ms=01091gitlab # [ 10.016059] gitaly[905]: time="2026-09-14T09:27:43.296Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9051092gitlab # [ 10.016554] gitaly[905]: time="2026-09-14T09:27:43.296Z" level=info msg="Started Gitaly" duration_ms=234 pid=9051093gitlab # [ 10.017241] gitaly[905]: time="2026-09-14T09:27:43.296Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9051094gitlab # [ 10.017978] gitaly[905]: time="2026-09-14T09:27:43.297Z" level=info msg="maintenance: daily scheduled" pid=905 scheduled="2026-09-14 12:00:00 +0000 UTC"1095gitlab # [ 10.724393] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1096gitlab # [ 10.727155] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1097gitlab # [ 10.756950] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1098gitlab # [ 11.930532] gitaly[905]: time="2026-09-14T09:27:45.212Z" level=info msg="License database preloaded" duration_ms=2150 pid=9051099gitlab # [ 13.401312] dhcpcd[771]: eth0: leased 10.0.2.15 for 86400 seconds1100gitlab # [ 13.402110] dhcpcd[771]: eth0: adding route to 10.0.2.0/241101gitlab # [ 13.402588] dhcpcd[771]: eth0: adding default route via 10.0.2.21102gitlab # [ 13.511211] systemd[1]: Started DHCP Client.1103gitlab # [ 13.511993] systemd[1]: Reached target Network is Online.1104gitlab # [ 40.724413] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: W, [2026-09-14T09:28:14.006430 #918] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1105gitlab # [ 42.862304] postgres[1315]: [1315] ERROR: relation "feature_gates" does not exist at character 5231106gitlab # [ 42.862883] postgres[1315]: [1315] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1107gitlab # [ 42.863741] postgres[1315]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1108gitlab # [ 42.866329] postgres[1315]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1109gitlab # [ 42.867275] postgres[1315]: attidentity AS identity,1110gitlab # [ 42.868360] postgres[1315]: attgenerated as attgenerated1111gitlab # [ 42.869979] postgres[1315]: FROM pg_attribute a1112gitlab # [ 42.871778] postgres[1315]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1113gitlab # [ 42.873404] postgres[1315]: LEFT JOIN pg_type t ON a.atttypid = t.oid1114gitlab # [ 42.874899] postgres[1315]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1115gitlab # [ 42.876423] postgres[1315]: WHERE a.attrelid = '"feature_gates"'::regclass1116gitlab # [ 42.877947] postgres[1315]: AND a.attnum > 0 AND NOT a.attisdropped1117gitlab # [ 42.879733] postgres[1315]: ORDER BY a.attnum1118gitlab # [ 42.881332] postgres[1315]:1119gitlab # [ 46.158716] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1348]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1120gitlab # [ 46.159309] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1348]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1121gitlab # [ 48.235062] gitlab-workhorse[832]: time="2026-09-14T09:28:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 58.245416] gitlab-workhorse[832]: time="2026-09-14T09:28:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1123gitlab # [ 61.487862] postgres[1423]: [1423] ERROR: relation "table_without_models" does not exist1124gitlab # [ 61.488283] postgres[1423]: [1423] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1125gitlab # [ 68.255060] gitlab-workhorse[832]: time="2026-09-14T09:28:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1126gitlab # [ 73.245737] systemd[1]: Finished gitlab-db-config.service.1127gitlab # [ 73.251631] systemd[1]: Started gitlab-sidekiq.service.1128gitlab # [ 73.256705] systemd[1]: Starting gitlab.service...1129gitlab # [ 73.601215] bundle[1590]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1130gitlab # [ 73.602160] bundle[1590]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1131gitlab # [ 73.603076] bundle[1590]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1132gitlab # [ 76.348994] sidekiq-cluster[1589]: {"severity":"INFO","time":"2026-09-14T09:28:49.614Z","message":"Starting cluster with 1 processes"}1133gitlab # [ 76.428916] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.691Z","pid":1590,"message":"* Enabling systemd notification integration"}1134gitlab # [ 76.429369] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"Puma starting in cluster mode..."}1135gitlab # [ 76.430558] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1136gitlab # [ 76.432311] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1137gitlab # [ 76.433625] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Min threads: 0"}1138gitlab # [ 76.434405] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Max threads: 4"}1139gitlab # [ 76.435975] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Environment: production"}1140gitlab # [ 76.438123] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Master PID: 1590"}1141gitlab # [ 76.439419] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Workers: 2"}1142gitlab # [ 76.440246] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1143gitlab # [ 76.441580] bundle[1590]: {"timestamp":"2026-09-14T09:28:49.711Z","pid":1590,"message":"* Preloading application"}1144gitlab # [ 78.265368] gitlab-workhorse[832]: time="2026-09-14T09:28:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 88.275562] gitlab-workhorse[832]: time="2026-09-14T09:29:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 98.286368] gitlab-workhorse[832]: time="2026-09-14T09:29:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147gitlab # [ 99.040512] bundle[1590]: W, [2026-09-14T09:29:12.322613 #1590] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1148gitlab # [ 101.353919] sidekiq-cluster[1620]: 2026-09-14T09:29:14.635Z pid=1620 tid=wl8 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1620, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433410048}1149gitlab # [ 101.524228] sidekiq-cluster[1620]: W, [2026-09-14T09:29:14.806860 #1620] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1150gitlab # [ 101.562256] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:14.844Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1151gitlab # [ 103.422644] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.704Z","message":"GitLab reliable fetch activated!"}1152gitlab # [ 103.425788] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.704Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1153gitlab # [ 103.444680] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.727Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1154gitlab # [ 103.448893] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.729Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1155gitlab # [ 103.453174] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.731Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1156gitlab # [ 103.457162] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.734Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1157gitlab # [ 103.461577] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.736Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1158gitlab # [ 103.465672] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.738Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1159gitlab # [ 103.469855] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.740Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1160gitlab # [ 103.473641] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.742Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1161gitlab # [ 103.477437] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.744Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1162gitlab # [ 103.481275] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.746Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1163gitlab # [ 103.485610] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.748Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1164gitlab # [ 103.489091] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.750Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1165gitlab # [ 103.491130] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.752Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1166gitlab # [ 103.493402] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.754Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1167gitlab # [ 103.495546] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.756Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1168gitlab # [ 103.497759] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.758Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1169gitlab # [ 103.499814] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.760Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1170gitlab # [ 103.501831] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.763Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1171gitlab # [ 103.504081] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.765Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1172gitlab # [ 103.506164] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.768Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1173gitlab # [ 103.508493] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.770Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1174gitlab # [ 103.511026] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.773Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1175gitlab # [ 103.513127] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.775Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1176gitlab # [ 103.515350] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.778Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1177gitlab # [ 103.517524] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.780Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1178gitlab # [ 103.519685] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.783Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1179gitlab # [ 103.521772] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.785Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1180gitlab # [ 103.523912] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.788Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1181gitlab # [ 103.526064] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.790Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1182gitlab # [ 103.528257] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.793Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1183gitlab # [ 103.530340] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.795Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1184gitlab # [ 103.532778] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.798Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1185gitlab # [ 103.534964] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.800Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1186gitlab # [ 103.537242] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.803Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1187gitlab # [ 103.539519] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.805Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1188gitlab # [ 103.541761] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.807Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1189gitlab # [ 103.543938] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.810Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1190gitlab # [ 103.546027] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.812Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1191gitlab # [ 103.548176] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.814Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1192gitlab # [ 103.550241] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.817Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1193gitlab # [ 103.552479] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.819Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1194gitlab # [ 103.554662] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.821Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1195gitlab # [ 103.556859] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.823Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1196gitlab # [ 103.558932] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.825Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1197gitlab # [ 103.561198] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.828Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1198gitlab # [ 103.563433] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.830Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1199gitlab # [ 103.565448] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.832Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1200gitlab # [ 103.567496] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.834Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1201gitlab # [ 103.569543] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.836Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1202gitlab # [ 103.572048] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.838Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1203gitlab # [ 103.572371] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.840Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1204gitlab # [ 103.572858] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.843Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1205gitlab # [ 103.573613] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.853Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1206gitlab # [ 103.575034] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.857Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1207gitlab # [ 103.584442] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.867Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1208gitlab # [ 103.589392] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.872Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1209gitlab # [ 103.594758] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.877Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1210gitlab # [ 103.600382] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.883Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1211gitlab # [ 103.604867] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.887Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1212gitlab # [ 103.610171] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.893Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1213gitlab # [ 103.614899] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.897Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1214gitlab # [ 103.619674] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.902Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1215gitlab # [ 103.625614] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.908Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1216gitlab # [ 103.631601] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.914Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1217gitlab # [ 103.637227] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.920Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1218gitlab # [ 103.642997] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.925Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1219gitlab # [ 103.648073] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.930Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1220gitlab # [ 103.652918] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.935Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1221gitlab # [ 103.658194] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.941Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1222gitlab # [ 103.697548] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.980Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1223gitlab # [ 103.702109] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.984Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1224gitlab # [ 103.708807] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.991Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1225gitlab # [ 103.715628] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:16.998Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1226gitlab # [ 103.721074] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.003Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1227gitlab # [ 103.725290] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.008Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1228gitlab # [ 103.729775] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.012Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1229gitlab # [ 103.734622] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.017Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1230gitlab # [ 103.739454] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.022Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1231gitlab # [ 103.743729] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.026Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1232gitlab # [ 103.747969] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.030Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1233gitlab # [ 103.752436] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.035Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1234gitlab # [ 103.757109] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.039Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1235gitlab # [ 103.761972] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.044Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1236gitlab # [ 103.766483] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.049Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1237gitlab # [ 103.771286] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.054Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1238gitlab # [ 103.775819] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.058Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1239gitlab # [ 103.780888] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.063Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1240gitlab # [ 103.785354] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.068Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1241gitlab # [ 103.789829] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.072Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1242gitlab # [ 103.796993] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.079Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1243gitlab # [ 103.803481] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.086Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1244gitlab # [ 103.808727] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.091Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1245gitlab # [ 103.814880] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.097Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1246gitlab # [ 103.821255] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.104Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1247gitlab # [ 103.829246] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.112Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1248gitlab # [ 103.835901] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.118Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1249gitlab # [ 103.842873] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.125Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1250gitlab # [ 103.848341] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.131Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1251gitlab # [ 103.869809] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.152Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1252gitlab # [ 103.873643] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.156Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1253gitlab # [ 103.879796] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.162Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1254gitlab # [ 103.890107] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.172Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1255gitlab # [ 103.899235] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.182Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1256gitlab # [ 103.907551] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.190Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1257gitlab # [ 103.917219] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.199Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1258gitlab # [ 103.924876] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.207Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1259gitlab # [ 103.932899] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.215Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1260gitlab # [ 103.942946] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.225Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1261gitlab # [ 103.950860] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:17.233Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1262gitlab # [ 108.287188] gitlab-workhorse[832]: time="2026-09-14T09:29:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1263gitlab # [ 118.294692] gitlab-workhorse[832]: time="2026-09-14T09:29:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1264gitlab # [ 127.984140] postgres[2058]: [2058] ERROR: relation "table_without_models" does not exist1265gitlab # [ 127.985464] postgres[2058]: [2058] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1266gitlab # [ 128.295576] gitlab-workhorse[832]: time="2026-09-14T09:29:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1267gitlab # [ 129.027048] postgres[2078]: [2078] ERROR: relation "table_without_models" does not exist1268gitlab # [ 129.027264] postgres[2078]: [2078] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1269gitlab # [ 130.571467] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:43.853Z","message":"Booted Rails 7.2.3.1 application in production environment"}1270gitlab # [ 130.576543] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:43.859Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1271gitlab # [ 130.576819] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:43.859Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1272gitlab # [ 130.578057] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:43.859Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1273gitlab # [ 130.591677] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:43.874Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1274gitlab # [ 131.769052] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.051Z","pid":1590,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1275gitlab # [ 131.769267] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.051Z","pid":1590,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1276gitlab # [ 131.770152] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.051Z","pid":1590,"message":"! #\u003cThread:0x000071620918ff88 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1277gitlab # [ 131.770934] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.051Z","pid":1590,"message":"! #\u003cThread:0x000071620943d578 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1278gitlab # [ 131.771782] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.051Z","pid":1590,"message":"Use Ctrl-C to stop"}1279gitlab # [ 131.804447] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.086Z","pid":1590,"message":"Sending status to systemd every 1.0 sec"}1280gitlab # [ 131.864128] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:45.146Z","message":"Cleaning working queues"}1281gitlab # [ 131.869482] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:45.152Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3fc04e2f0d0878ca10f1e31d","created_at":"2026-09-14T09:28:46.179Z","trace_propagation_headers":{"sentry-trace":"80ef421810f440f3ae06279f75aaabf5-1a627bcede454b30","baggage":"sentry-trace_id=80ef421810f440f3ae06279f75aaabf5,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d58849c73c7b79951abe2538888e2845","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T09:28:46.187Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3fc04e2f0d0878ca10f1e31d: start","job_status":"start","queue_duration_s":58.965036,"scheduling_latency_s":58.965083}1282gitlab # [ 132.029709] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:29:45.312Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3fc04e2f0d0878ca10f1e31d","created_at":"2026-09-14T09:28:46.179Z","trace_propagation_headers":{"sentry-trace":"80ef421810f440f3ae06279f75aaabf5-1a627bcede454b30","baggage":"sentry-trace_id=80ef421810f440f3ae06279f75aaabf5,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d58849c73c7b79951abe2538888e2845","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T09:28:46.187Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3fc04e2f0d0878ca10f1e31d: done: 0.160275 sec","job_status":"done","queue_duration_s":58.965036,"scheduling_latency_s":58.965083,"redis_calls":26,"redis_duration_s":0.004969,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002758,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.081388837,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.160275,"completed_at":"2026-09-14T09:29:45.312Z","load_balancing_strategy":"primary","db_duration_s":0.05145,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1283gitlab # [ 132.105215] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.386Z","pid":1590,"message":"- Worker 0 (PID: 2162) booted in 0.27s, phase: 0"}1284gitlab # [ 132.131519] bundle[1590]: {"timestamp":"2026-09-14T09:29:45.414Z","pid":1590,"message":"- Worker 1 (PID: 2166) booted in 0.28s, phase: 0"}1285gitlab # [ 132.135107] systemd[1]: Started gitlab.service.1286gitlab # [ 132.135932] systemd[1]: Reached target Common target for all GitLab services..1287gitlab # [ 132.139231] systemd[1]: Starting setup-gitlab.service...1288gitlab # [ 132.156660] setup-gitlab-start[2197]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1289gitlab # [ 134.394412] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:29:47 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 22151290gitlab # [ 134.403154] setup-gitlab-start[2197]: + TOKEN=nixbot-test-token-of-20-chars1291gitlab # [ 134.403336] setup-gitlab-start[2197]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1292gitlab # [ 134.428382] runuser[2219]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1293gitlab # [ 134.814549] setup-gitlab-start[2220]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1294gitlab # [ 134.814789] setup-gitlab-start[2220]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1295gitlab # [ 134.816073] setup-gitlab-start[2220]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1296gitlab # [ 135.392399] setup-gitlab-start[2220]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1297gitlab # [ 135.392755] setup-gitlab-start[2220]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1298gitlab # [ 135.394520] setup-gitlab-start[2220]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1299gitlab # [ 149.867774] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"dc904a05fb96d40423ae73a6","created_at":"2026-09-14T09:30:03.097Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c497af688040b9469ed20d3307679a00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.145Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-dc904a05fb96d40423ae73a6: start","job_status":"start","queue_duration_s":0.003524,"scheduling_latency_s":0.003582}1300gitlab # [ 149.949836] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.232Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1301gitlab # [ 149.981129] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"dc904a05fb96d40423ae73a6","created_at":"2026-09-14T09:30:03.097Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c497af688040b9469ed20d3307679a00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.145Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-dc904a05fb96d40423ae73a6: done: 0.114073 sec","job_status":"done","queue_duration_s":0.003524,"scheduling_latency_s":0.003582,"redis_calls":10,"redis_duration_s":0.011202,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002695,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004508,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000803175,"duration_s":0.114073,"completed_at":"2026-09-14T09:30:03.263Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 150.054483] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"c8df8d2535abb8da82c94dc9","created_at":"2026-09-14T09:30:03.182Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0742dffac325abaf5b7df49c11f421a","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.199Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-c8df8d2535abb8da82c94dc9: start","job_status":"start","queue_duration_s":0.13708,"scheduling_latency_s":0.137112}1303gitlab # [ 150.132511] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Export::PruneProjectExportJobsWorker","jid":"c8df8d2535abb8da82c94dc9","created_at":"2026-09-14T09:30:03.182Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0742dffac325abaf5b7df49c11f421a","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b2f7045536fe38235224f4d98e515e25b0b0eea1aa1ae4b4ab9f99ab22c53b43","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.199Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Export::PruneProjectExportJobsWorker JID-c8df8d2535abb8da82c94dc9: done: 0.077599 sec","job_status":"done","queue_duration_s":0.13708,"scheduling_latency_s":0.137112,"redis_calls":10,"redis_duration_s":0.009438,"redis_read_bytes":9,"redis_write_bytes":1173,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001727,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":236,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000602312,"duration_s":0.077599,"completed_at":"2026-09-14T09:30:03.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 150.186785] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"57ae5add40f3012db31c9a53","created_at":"2026-09-14T09:30:03.220Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d543371bdc08f1bd25ae0d8f40c94e3","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.241Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-57ae5add40f3012db31c9a53: start","job_status":"start","queue_duration_s":0.227806,"scheduling_latency_s":0.227858}1305gitlab # [ 150.313576] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"57ae5add40f3012db31c9a53","created_at":"2026-09-14T09:30:03.220Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d543371bdc08f1bd25ae0d8f40c94e3","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.241Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-57ae5add40f3012db31c9a53: done: 0.126741 sec","job_status":"done","queue_duration_s":0.227806,"scheduling_latency_s":0.227858,"redis_calls":10,"redis_duration_s":0.013306,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004583,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001006832,"duration_s":0.126741,"completed_at":"2026-09-14T09:30:03.595Z","load_balancing_strategy":"primary","db_duration_s":0.058156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 150.383861] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"eb5850c06d500adf855aa458","created_at":"2026-09-14T09:30:03.259Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d815fb1ab445d628167559f557d2f538","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.336Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-eb5850c06d500adf855aa458: start","job_status":"start","queue_duration_s":0.330117,"scheduling_latency_s":0.33015}1307gitlab # [ 150.446864] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckExportJobsWorker","jid":"eb5850c06d500adf855aa458","created_at":"2026-09-14T09:30:03.259Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d815fb1ab445d628167559f557d2f538","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c05136a1912f23c3950f9f6243170d655fcc26d1e5b96896d7e248d72f9039fe","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.336Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckExportJobsWorker JID-eb5850c06d500adf855aa458: done: 0.062918 sec","job_status":"done","queue_duration_s":0.330117,"scheduling_latency_s":0.33015,"redis_calls":10,"redis_duration_s":0.017022,"redis_read_bytes":9,"redis_write_bytes":995,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002356,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00618,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":192,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002434946,"duration_s":0.062918,"completed_at":"2026-09-14T09:30:03.729Z","load_balancing_strategy":"primary","db_duration_s":0.007559,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 150.500078] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.782Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"4dfe6b1a79eb0b948d88973b","created_at":"2026-09-14T09:30:03.366Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f1fe12d5a85c6b05cb78cb6885932f2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.412Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-4dfe6b1a79eb0b948d88973b: start","job_status":"start","queue_duration_s":0.370421,"scheduling_latency_s":0.370471}1309gitlab # [ 150.611659] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.893Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"4dfe6b1a79eb0b948d88973b","created_at":"2026-09-14T09:30:03.366Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f1fe12d5a85c6b05cb78cb6885932f2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.412Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-4dfe6b1a79eb0b948d88973b: done: 0.111514 sec","job_status":"done","queue_duration_s":0.370421,"scheduling_latency_s":0.370471,"redis_calls":16,"redis_duration_s":0.032547,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.025983,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001825,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001385651,"duration_s":0.111514,"completed_at":"2026-09-14T09:30:03.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 150.680330] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:03.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"223cf39853fe2ccca70f88a3","created_at":"2026-09-14T09:30:03.478Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9ca92f17b2fdb4e49b3f3828d56e6c4","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.490Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-223cf39853fe2ccca70f88a3: start","job_status":"start","queue_duration_s":0.471959,"scheduling_latency_s":0.472013}1311gitlab # [ 150.796692] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"223cf39853fe2ccca70f88a3","created_at":"2026-09-14T09:30:03.478Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9ca92f17b2fdb4e49b3f3828d56e6c4","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.490Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-223cf39853fe2ccca70f88a3: done: 0.116611 sec","job_status":"done","queue_duration_s":0.471959,"scheduling_latency_s":0.472013,"redis_calls":10,"redis_duration_s":0.015638,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004539,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003178,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212318,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.005389791},"duration_s":0.116611,"completed_at":"2026-09-14T09:30:04.079Z","load_balancing_strategy":"primary","db_duration_s":0.055944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 150.802549] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6b7ac918fd4db4b936813cfa","created_at":"2026-09-14T09:30:03.506Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84cd91664e79959e4ea39f368244c215","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.584Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6b7ac918fd4db4b936813cfa: start","job_status":"start","queue_duration_s":0.500918,"scheduling_latency_s":0.500968}1313gitlab # [ 150.867084] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6b7ac918fd4db4b936813cfa","created_at":"2026-09-14T09:30:03.506Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84cd91664e79959e4ea39f368244c215","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.584Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6b7ac918fd4db4b936813cfa: done: 0.03111 sec","job_status":"done","queue_duration_s":0.500918,"scheduling_latency_s":0.500968,"redis_calls":12,"redis_duration_s":0.002858,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000645333,"exclusive_lock_hold_duration_s":0.007670527999999999,"duration_s":0.03111,"completed_at":"2026-09-14T09:30:04.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004863,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 150.875719] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"1b8d5cfe99e16c08a000c37c","created_at":"2026-09-14T09:30:03.660Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42290941a5d30014308bcbf6d79507fa","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.671Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-1b8d5cfe99e16c08a000c37c: start","job_status":"start","queue_duration_s":0.44558,"scheduling_latency_s":0.445597}1315gitlab # [ 150.916355] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"1b8d5cfe99e16c08a000c37c","created_at":"2026-09-14T09:30:03.660Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42290941a5d30014308bcbf6d79507fa","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.671Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-1b8d5cfe99e16c08a000c37c: done: 0.020009 sec","job_status":"done","queue_duration_s":0.44558,"scheduling_latency_s":0.445597,"redis_calls":10,"redis_duration_s":0.001921,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000300597,"duration_s":0.020009,"completed_at":"2026-09-14T09:30:04.137Z","load_balancing_strategy":"primary","db_duration_s":0.004495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 150.925056] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"37a3b178ed7e2b757218c5f2","created_at":"2026-09-14T09:30:03.687Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b284aa453ce247751645204446f151d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.698Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-37a3b178ed7e2b757218c5f2: start","job_status":"start","queue_duration_s":0.439475,"scheduling_latency_s":0.439489}1317gitlab # [ 150.938780] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.146Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1318gitlab # [ 150.942777] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"37a3b178ed7e2b757218c5f2","created_at":"2026-09-14T09:30:03.687Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b284aa453ce247751645204446f151d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.698Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-37a3b178ed7e2b757218c5f2: done: 0.012699 sec","job_status":"done","queue_duration_s":0.439475,"scheduling_latency_s":0.439489,"redis_calls":10,"redis_duration_s":0.002477,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001112,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025087,"duration_s":0.012699,"completed_at":"2026-09-14T09:30:04.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1319gitlab # [ 150.993354] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"c62f55772b06aee899a64efe","created_at":"2026-09-14T09:30:03.717Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"138752619a627a52de449cd13b288c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.780Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-c62f55772b06aee899a64efe: start","job_status":"start","queue_duration_s":0.371664,"scheduling_latency_s":0.371678}1320gitlab # [ 151.034042] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"c62f55772b06aee899a64efe","created_at":"2026-09-14T09:30:03.717Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"138752619a627a52de449cd13b288c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.780Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-c62f55772b06aee899a64efe: done: 0.021368 sec","job_status":"done","queue_duration_s":0.371664,"scheduling_latency_s":0.371678,"redis_calls":20,"redis_duration_s":0.002839,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000807,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000401,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000299479,"duration_s":0.021368,"completed_at":"2026-09-14T09:30:04.173Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 151.036043] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c919d03ea0d9fef748826624","created_at":"2026-09-14T09:30:03.803Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b853096d260e44d2e469134be10216f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.815Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c919d03ea0d9fef748826624: start","job_status":"start","queue_duration_s":0.359322,"scheduling_latency_s":0.359337}1322gitlab # [ 151.050033] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.188Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c919d03ea0d9fef748826624","created_at":"2026-09-14T09:30:03.803Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b853096d260e44d2e469134be10216f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.815Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c919d03ea0d9fef748826624: done: 0.014434 sec","job_status":"done","queue_duration_s":0.359322,"scheduling_latency_s":0.359337,"redis_calls":12,"redis_duration_s":0.0024690000000000003,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000362,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000633321,"exclusive_lock_hold_duration_s":0.002549486,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.014434,"completed_at":"2026-09-14T09:30:04.188Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 151.116256] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.189Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bb9ecef944121f0d5e371957","created_at":"2026-09-14T09:30:03.823Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11e85ba22522988891ea58befa784d45","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.834Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bb9ecef944121f0d5e371957: start","job_status":"start","queue_duration_s":0.355434,"scheduling_latency_s":0.355448}1324gitlab # [ 151.127338] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bb9ecef944121f0d5e371957","created_at":"2026-09-14T09:30:03.823Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11e85ba22522988891ea58befa784d45","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.834Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bb9ecef944121f0d5e371957: done: 0.220168 sec","job_status":"done","queue_duration_s":0.355434,"scheduling_latency_s":0.355448,"redis_calls":565,"redis_duration_s":0.06813200000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.066683,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.17125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000264838,"duration_s":0.220168,"completed_at":"2026-09-14T09:30:04.409Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 151.178839] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ed6441d6cc41ec9ae641ad8e","created_at":"2026-09-14T09:30:03.838Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99a33340d386a39fb2ddf4cfb1fdce6c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.850Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ed6441d6cc41ec9ae641ad8e: start","job_status":"start","queue_duration_s":0.560511,"scheduling_latency_s":0.560527}1326gitlab # [ 151.179743] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ed6441d6cc41ec9ae641ad8e","created_at":"2026-09-14T09:30:03.838Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99a33340d386a39fb2ddf4cfb1fdce6c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.850Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ed6441d6cc41ec9ae641ad8e: done: 0.018216 sec","job_status":"done","queue_duration_s":0.560511,"scheduling_latency_s":0.560527,"redis_calls":10,"redis_duration_s":0.001326,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210082,"duration_s":0.018216,"completed_at":"2026-09-14T09:30:04.429Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 151.194720] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"48c097ced1495a00edc4e8ce","created_at":"2026-09-14T09:30:03.879Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb904b2629d5dfd81f8abab560900e6b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.895Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-48c097ced1495a00edc4e8ce: start","job_status":"start","queue_duration_s":0.534507,"scheduling_latency_s":0.534538}1328gitlab # [ 151.208070] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.437Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1329gitlab # [ 151.211688] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"48c097ced1495a00edc4e8ce","created_at":"2026-09-14T09:30:03.879Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb904b2629d5dfd81f8abab560900e6b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.895Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-48c097ced1495a00edc4e8ce: done: 0.01101 sec","job_status":"done","queue_duration_s":0.534507,"scheduling_latency_s":0.534538,"redis_calls":10,"redis_duration_s":0.001403,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205613,"duration_s":0.01101,"completed_at":"2026-09-14T09:30:04.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 151.264243] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"f854e985ed54d723d01c18cf","created_at":"2026-09-14T09:30:03.972Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a0f91610af15b438165060afeb902f","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.992Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-f854e985ed54d723d01c18cf: start","job_status":"start","queue_duration_s":0.450042,"scheduling_latency_s":0.450056}1331gitlab # [ 151.264940] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"f854e985ed54d723d01c18cf","created_at":"2026-09-14T09:30:03.972Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a0f91610af15b438165060afeb902f","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:03.992Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-f854e985ed54d723d01c18cf: done: 0.011343 sec","job_status":"done","queue_duration_s":0.450042,"scheduling_latency_s":0.450056,"redis_calls":11,"redis_duration_s":0.001505,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204215,"duration_s":0.011343,"completed_at":"2026-09-14T09:30:04.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 151.319823] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1d45365e0c02b239be7cef8a","created_at":"2026-09-14T09:30:04.018Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f849cec3e5d1178f22330ab596774f73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.032Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1d45365e0c02b239be7cef8a: start","job_status":"start","queue_duration_s":0.42192,"scheduling_latency_s":0.421934}1333gitlab # [ 151.332988] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1d45365e0c02b239be7cef8a","created_at":"2026-09-14T09:30:04.018Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f849cec3e5d1178f22330ab596774f73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.032Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1d45365e0c02b239be7cef8a: done: 0.16094 sec","job_status":"done","queue_duration_s":0.42192,"scheduling_latency_s":0.421934,"redis_calls":14,"redis_duration_s":0.002392,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000336915,"exclusive_lock_hold_duration_s":0.142502749,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.16094,"completed_at":"2026-09-14T09:30:04.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 151.355231] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"6e9cb9a14fa57bd470fe72ee","created_at":"2026-09-14T09:30:04.051Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eebab14046a292089c1c602993102372","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.055Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-6e9cb9a14fa57bd470fe72ee: start","job_status":"start","queue_duration_s":0.561003,"scheduling_latency_s":0.561019}1335gitlab # [ 151.394606] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"6e9cb9a14fa57bd470fe72ee","created_at":"2026-09-14T09:30:04.051Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eebab14046a292089c1c602993102372","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.055Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-6e9cb9a14fa57bd470fe72ee: done: 0.015784 sec","job_status":"done","queue_duration_s":0.561003,"scheduling_latency_s":0.561019,"redis_calls":11,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000312,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000229917,"duration_s":0.015784,"completed_at":"2026-09-14T09:30:04.632Z","load_balancing_strategy":"primary","db_duration_s":0.001852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 151.433908] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"7768839846c7fd65b5ba5d6c","created_at":"2026-09-14T09:30:04.057Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cb79478970d2bbb97c0f0677d84e44b","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.060Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-7768839846c7fd65b5ba5d6c: start","job_status":"start","queue_duration_s":0.57332,"scheduling_latency_s":0.573334}1337gitlab # [ 151.434750] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.648Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"7768839846c7fd65b5ba5d6c","created_at":"2026-09-14T09:30:04.057Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cb79478970d2bbb97c0f0677d84e44b","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.060Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-7768839846c7fd65b5ba5d6c: done: 0.014816 sec","job_status":"done","queue_duration_s":0.57332,"scheduling_latency_s":0.573334,"redis_calls":13,"redis_duration_s":0.001658,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000672,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000201981,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.014816,"completed_at":"2026-09-14T09:30:04.648Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001733,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 151.457077] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"551019c416555ea7146733e0","created_at":"2026-09-14T09:30:04.061Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42a4bafc0b0e3bd7d9f7d32779273fbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.071Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-551019c416555ea7146733e0: start","job_status":"start","queue_duration_s":0.577345,"scheduling_latency_s":0.577359}1339gitlab # [ 151.470979] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"551019c416555ea7146733e0","created_at":"2026-09-14T09:30:04.061Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42a4bafc0b0e3bd7d9f7d32779273fbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.071Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-551019c416555ea7146733e0: done: 0.075287 sec","job_status":"done","queue_duration_s":0.577345,"scheduling_latency_s":0.577359,"redis_calls":14,"redis_duration_s":0.002192,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000549,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000806,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000308419,"exclusive_lock_hold_duration_s":0.061391880999999995,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.075287,"completed_at":"2026-09-14T09:30:04.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 151.530710] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.725Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"cbebe34d50b6e6965fcd7932","created_at":"2026-09-14T09:30:04.159Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-ad24eaf4a83b4a98","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"138752619a627a52de449cd13b288c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.164Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-cbebe34d50b6e6965fcd7932: start","job_status":"start","queue_duration_s":0.560524,"scheduling_latency_s":0.560539}1341gitlab # [ 151.531703] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.740Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"cbebe34d50b6e6965fcd7932","created_at":"2026-09-14T09:30:04.159Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-ad24eaf4a83b4a98","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"138752619a627a52de449cd13b288c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.164Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-cbebe34d50b6e6965fcd7932: done: 0.015192 sec","job_status":"done","queue_duration_s":0.560524,"scheduling_latency_s":0.560539,"redis_calls":10,"redis_duration_s":0.001271,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194997,"duration_s":0.015192,"completed_at":"2026-09-14T09:30:04.740Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 151.570605] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.741Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"0134aed71e8b2a77a491a506","created_at":"2026-09-14T09:30:04.165Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-ad24eaf4a83b4a98","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"138752619a627a52de449cd13b288c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.167Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-0134aed71e8b2a77a491a506: start","job_status":"start","queue_duration_s":0.573756,"scheduling_latency_s":0.573769}1343gitlab # [ 151.574566] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.754Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"0134aed71e8b2a77a491a506","created_at":"2026-09-14T09:30:04.165Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-ad24eaf4a83b4a98","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"138752619a627a52de449cd13b288c7e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.167Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-0134aed71e8b2a77a491a506: done: 0.01248 sec","job_status":"done","queue_duration_s":0.573756,"scheduling_latency_s":0.573769,"redis_calls":10,"redis_duration_s":0.001319,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199188,"duration_s":0.01248,"completed_at":"2026-09-14T09:30:04.754Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 151.656761] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:04.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"76d147779a9bb8afd07df2ce","created_at":"2026-09-14T09:30:04.082Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.937Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-76d147779a9bb8afd07df2ce: start","job_status":"start","queue_duration_s":0.001676,"scheduling_latency_s":0.001711}1345gitlab # [ 151.754148] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"76d147779a9bb8afd07df2ce","created_at":"2026-09-14T09:30:04.082Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:04.937Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-76d147779a9bb8afd07df2ce: done: 0.097202 sec","job_status":"done","queue_duration_s":0.001676,"scheduling_latency_s":0.001711,"redis_calls":19,"redis_duration_s":0.026341999999999997,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004501,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000514,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.017341,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003986,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001999696,"duration_s":0.097202,"completed_at":"2026-09-14T09:30:05.036Z","load_balancing_strategy":"primary","db_duration_s":0.003724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 151.776505] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1448014565f5c62ebe01c6d1","created_at":"2026-09-14T09:30:04.960Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"755ec1749135876a8f21b9764fd7b2f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T09:30:04.975Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1448014565f5c62ebe01c6d1: start","job_status":"start","queue_duration_s":0.067499,"scheduling_latency_s":0.067547}1347gitlab # [ 151.781745] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.064Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1348gitlab # [ 151.788772] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1448014565f5c62ebe01c6d1","created_at":"2026-09-14T09:30:04.960Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"755ec1749135876a8f21b9764fd7b2f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T09:30:04.975Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1448014565f5c62ebe01c6d1: done: 0.029035 sec","job_status":"done","queue_duration_s":0.067499,"scheduling_latency_s":0.067547,"redis_calls":10,"redis_duration_s":0.007809,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002808,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000439162,"duration_s":0.029035,"completed_at":"2026-09-14T09:30:05.071Z","load_balancing_strategy":"primary","db_duration_s":0.000842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 151.807421] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c990a8792e1219a85b36ee36","created_at":"2026-09-14T09:30:04.991Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a89e8789b7c3b7df3faca62c5c24c866","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.007Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c990a8792e1219a85b36ee36: start","job_status":"start","queue_duration_s":0.064509,"scheduling_latency_s":0.064524}1350gitlab # [ 151.855477] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c990a8792e1219a85b36ee36","created_at":"2026-09-14T09:30:04.991Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a89e8789b7c3b7df3faca62c5c24c866","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.007Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c990a8792e1219a85b36ee36: done: 0.021803 sec","job_status":"done","queue_duration_s":0.064509,"scheduling_latency_s":0.064524,"redis_calls":13,"redis_duration_s":0.00184,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0077797610000000005,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021803,"completed_at":"2026-09-14T09:30:05.094Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004709,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1351gitlab # [ 151.901200] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.095Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a1ebb26b545e0942f8dca795","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a1ebb26b545e0942f8dca795: start","job_status":"start","queue_duration_s":0.083058,"scheduling_latency_s":0.083072}1352gitlab # [ 151.901993] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.105Z","message":"No existing merge request to be cleaned up."}1353gitlab # [ 151.902865] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.115Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a1ebb26b545e0942f8dca795","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a1ebb26b545e0942f8dca795: done: 0.020448 sec","job_status":"done","queue_duration_s":0.083058,"scheduling_latency_s":0.083072,"redis_calls":13,"redis_duration_s":0.0026190000000000002,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000257295,"duration_s":0.020448,"completed_at":"2026-09-14T09:30:05.115Z","load_balancing_strategy":"primary","db_duration_s":0.004974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 151.912190] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.116Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d1f48ed19b0f0fd1f2de564d","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d1f48ed19b0f0fd1f2de564d: start","job_status":"start","queue_duration_s":0.104617,"scheduling_latency_s":0.104632}1355gitlab # [ 151.960190] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.123Z","message":"No existing merge request to be cleaned up."}1356gitlab # [ 151.960911] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.129Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d1f48ed19b0f0fd1f2de564d","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d1f48ed19b0f0fd1f2de564d: done: 0.012278 sec","job_status":"done","queue_duration_s":0.104617,"scheduling_latency_s":0.104632,"redis_calls":10,"redis_duration_s":0.0019779999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001199,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012278,"completed_at":"2026-09-14T09:30:05.128Z","load_balancing_strategy":"primary","db_duration_s":0.00156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 151.961898] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.130Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"41db5de3e3bf7db53125cb69","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-41db5de3e3bf7db53125cb69: start","job_status":"start","queue_duration_s":0.117919,"scheduling_latency_s":0.117962}1358gitlab # [ 151.969542] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.137Z","message":"No existing merge request to be cleaned up."}1359gitlab # [ 151.971041] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.147Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"41db5de3e3bf7db53125cb69","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-41db5de3e3bf7db53125cb69: done: 0.017855 sec","job_status":"done","queue_duration_s":0.117919,"scheduling_latency_s":0.117962,"redis_calls":10,"redis_duration_s":0.005562,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004688,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017855,"completed_at":"2026-09-14T09:30:05.147Z","load_balancing_strategy":"primary","db_duration_s":0.003785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 152.006739] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.148Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6c09c9d34a1dd8bc3b96fb9a","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6c09c9d34a1dd8bc3b96fb9a: start","job_status":"start","queue_duration_s":0.13682,"scheduling_latency_s":0.136834}1361gitlab # [ 152.017897] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.166Z","message":"No existing merge request to be cleaned up."}1362gitlab # [ 152.019490] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.171Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6c09c9d34a1dd8bc3b96fb9a","created_at":"2026-09-14T09:30:04.988Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-98565f0f79004e47","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"85c9c673df6189d2f62df3a5c0f82469","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.012Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6c09c9d34a1dd8bc3b96fb9a: done: 0.022554 sec","job_status":"done","queue_duration_s":0.13682,"scheduling_latency_s":0.136834,"redis_calls":10,"redis_duration_s":0.011834,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.010208,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022554,"completed_at":"2026-09-14T09:30:05.171Z","load_balancing_strategy":"primary","db_duration_s":0.001717,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 152.063200] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"52b5e79be5be4eb9147be653","created_at":"2026-09-14T09:30:05.021Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d361efbb067c2996dd86ab12b06589f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.040Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-52b5e79be5be4eb9147be653: start","job_status":"start","queue_duration_s":0.132083,"scheduling_latency_s":0.132099}1364gitlab # [ 152.065642] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.186Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"52b5e79be5be4eb9147be653","created_at":"2026-09-14T09:30:05.021Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d361efbb067c2996dd86ab12b06589f","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.040Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-52b5e79be5be4eb9147be653: done: 0.013533 sec","job_status":"done","queue_duration_s":0.132083,"scheduling_latency_s":0.132099,"redis_calls":12,"redis_duration_s":0.002488,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000508,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000279086,"duration_s":0.013533,"completed_at":"2026-09-14T09:30:05.186Z","load_balancing_strategy":"primary","db_duration_s":0.000396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 152.149509] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.432Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8a8ba1445e6799cc308aefeb","created_at":"2026-09-14T09:30:05.054Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc4ec461234418be1a52733de5a0bd02","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.430Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8a8ba1445e6799cc308aefeb: start","job_status":"start","queue_duration_s":0.001339,"scheduling_latency_s":0.001373}1366gitlab # [ 152.196442] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.479Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1367gitlab # [ 152.221047] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8a8ba1445e6799cc308aefeb","created_at":"2026-09-14T09:30:05.054Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc4ec461234418be1a52733de5a0bd02","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.430Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8a8ba1445e6799cc308aefeb: done: 0.070865 sec","job_status":"done","queue_duration_s":0.001339,"scheduling_latency_s":0.001373,"redis_calls":10,"redis_duration_s":0.013323000000000002,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009501,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005610769,"duration_s":0.070865,"completed_at":"2026-09-14T09:30:05.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000695,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 152.242890] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.509Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"9db1878fc12be13ce78d4979","created_at":"2026-09-14T09:30:05.456Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.473Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-9db1878fc12be13ce78d4979: start","job_status":"start","queue_duration_s":0.036298,"scheduling_latency_s":0.03633}1369gitlab # [ 152.353258] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"9db1878fc12be13ce78d4979","created_at":"2026-09-14T09:30:05.456Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.473Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-9db1878fc12be13ce78d4979: done: 0.125972 sec","job_status":"done","queue_duration_s":0.036298,"scheduling_latency_s":0.03633,"redis_calls":24,"redis_duration_s":0.055279,"redis_read_bytes":24,"redis_write_bytes":5972,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003957,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000865,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3805,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.009853,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.040604,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375187,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.125972,"completed_at":"2026-09-14T09:30:05.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 152.413454] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e4b9e4ee216b39cf8dacbb1f","created_at":"2026-09-14T09:30:05.490Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"420394cf1d8cd049c1db3c78c27cec12","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.508Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e4b9e4ee216b39cf8dacbb1f: start","job_status":"start","queue_duration_s":0.187091,"scheduling_latency_s":0.187147}1371gitlab # [ 152.489682] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e4b9e4ee216b39cf8dacbb1f","created_at":"2026-09-14T09:30:05.490Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"420394cf1d8cd049c1db3c78c27cec12","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.508Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e4b9e4ee216b39cf8dacbb1f: done: 0.07628 sec","job_status":"done","queue_duration_s":0.187091,"scheduling_latency_s":0.187147,"redis_calls":10,"redis_duration_s":0.013779000000000001,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001935,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002624,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001252115,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.07628,"completed_at":"2026-09-14T09:30:05.771Z","load_balancing_strategy":"primary","db_duration_s":0.020239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 152.491592] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.774Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b79fb8ddb08e3ecd36b04916","created_at":"2026-09-14T09:30:05.557Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a76e85a383274664e60e3d38def88da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.568Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b79fb8ddb08e3ecd36b04916: start","job_status":"start","queue_duration_s":0.205088,"scheduling_latency_s":0.205119}1373gitlab # [ 152.547194] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b79fb8ddb08e3ecd36b04916","created_at":"2026-09-14T09:30:05.557Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a76e85a383274664e60e3d38def88da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.568Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b79fb8ddb08e3ecd36b04916: done: 0.021895 sec","job_status":"done","queue_duration_s":0.205088,"scheduling_latency_s":0.205119,"redis_calls":12,"redis_duration_s":0.00427,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002053,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002992,"duration_s":0.021895,"completed_at":"2026-09-14T09:30:05.795Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 152.548775] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"741d420bfaa5002208006912","created_at":"2026-09-14T09:30:05.577Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.586Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-741d420bfaa5002208006912: start","job_status":"start","queue_duration_s":0.210295,"scheduling_latency_s":0.21034}1375gitlab # [ 152.581136] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"741d420bfaa5002208006912","created_at":"2026-09-14T09:30:05.577Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.586Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-741d420bfaa5002208006912: done: 0.066719 sec","job_status":"done","queue_duration_s":0.210295,"scheduling_latency_s":0.21034,"redis_calls":71,"redis_duration_s":0.015106999999999999,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002499,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001252,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.010629,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000307022,"duration_s":0.066719,"completed_at":"2026-09-14T09:30:05.863Z","load_balancing_strategy":"primary","db_duration_s":0.001961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 152.622916] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"5513e77f3846533f2ba3f428","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-5513e77f3846533f2ba3f428: start","job_status":"start","queue_duration_s":0.251968,"scheduling_latency_s":0.251983}1377gitlab # [ 152.623658] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"5513e77f3846533f2ba3f428","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-5513e77f3846533f2ba3f428: done: 0.032172 sec","job_status":"done","queue_duration_s":0.251968,"scheduling_latency_s":0.251983,"redis_calls":18,"redis_duration_s":0.0047740000000000005,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002746,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000343061,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.032172,"completed_at":"2026-09-14T09:30:05.896Z","load_balancing_strategy":"primary","db_duration_s":0.011473,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 152.673880] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"8e1c1b0021b66742aa758466","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-8e1c1b0021b66742aa758466: start","job_status":"start","queue_duration_s":0.285347,"scheduling_latency_s":0.285361}1379gitlab # [ 152.674671] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"8e1c1b0021b66742aa758466","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-8e1c1b0021b66742aa758466: done: 0.02794 sec","job_status":"done","queue_duration_s":0.285347,"scheduling_latency_s":0.285361,"redis_calls":15,"redis_duration_s":0.0038770000000000002,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.002702,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.02794,"completed_at":"2026-09-14T09:30:05.926Z","load_balancing_strategy":"primary","db_duration_s":0.002619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 152.724660] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"fbbdac20ec1a59ed2e08e9fd","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-fbbdac20ec1a59ed2e08e9fd: start","job_status":"start","queue_duration_s":0.31428,"scheduling_latency_s":0.314296}1381gitlab # [ 152.725583] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"fbbdac20ec1a59ed2e08e9fd","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-fbbdac20ec1a59ed2e08e9fd: done: 0.012663 sec","job_status":"done","queue_duration_s":0.31428,"scheduling_latency_s":0.314296,"redis_calls":15,"redis_duration_s":0.001781,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":1,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":1,"duration_s":0.012663,"completed_at":"2026-09-14T09:30:05.939Z","load_balancing_strategy":"primary","db_duration_s":0.001256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 152.783134] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"c9cccf6206d419d17a7c7dcd","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-c9cccf6206d419d17a7c7dcd: start","job_status":"start","queue_duration_s":0.327773,"scheduling_latency_s":0.327787}1383gitlab # [ 152.783930] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"c9cccf6206d419d17a7c7dcd","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-c9cccf6206d419d17a7c7dcd: done: 0.012348 sec","job_status":"done","queue_duration_s":0.327773,"scheduling_latency_s":0.327787,"redis_calls":15,"redis_duration_s":0.001724,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.012348,"completed_at":"2026-09-14T09:30:05.952Z","load_balancing_strategy":"primary","db_duration_s":0.001201,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 152.845585] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"9a72b08fb41d8711b0ba2963","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-9a72b08fb41d8711b0ba2963: start","job_status":"start","queue_duration_s":0.340979,"scheduling_latency_s":0.340993}1385gitlab # [ 152.857459] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"9a72b08fb41d8711b0ba2963","created_at":"2026-09-14T09:30:05.580Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-b2a9ea853411405c","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"89f8580efc244a206c4a35a9a6842417","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.612Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-9a72b08fb41d8711b0ba2963: done: 0.012431 sec","job_status":"done","queue_duration_s":0.340979,"scheduling_latency_s":0.340993,"redis_calls":15,"redis_duration_s":0.0017569999999999999,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.012431,"completed_at":"2026-09-14T09:30:05.966Z","load_balancing_strategy":"primary","db_duration_s":0.001267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 152.858184] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"95632c5df754f91403bfaa35","created_at":"2026-09-14T09:30:05.606Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ee19eacf7a5e4420d912a73ab01e733","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.613Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-95632c5df754f91403bfaa35: start","job_status":"start","queue_duration_s":0.353484,"scheduling_latency_s":0.353497}1387gitlab # [ 152.905165] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"95632c5df754f91403bfaa35","created_at":"2026-09-14T09:30:05.606Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ee19eacf7a5e4420d912a73ab01e733","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.613Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-95632c5df754f91403bfaa35: done: 0.013621 sec","job_status":"done","queue_duration_s":0.353484,"scheduling_latency_s":0.353497,"redis_calls":10,"redis_duration_s":0.001447,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021539,"duration_s":0.013621,"completed_at":"2026-09-14T09:30:05.980Z","load_balancing_strategy":"primary","db_duration_s":0.000366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 152.906146] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"a3310d7cf4964ad6d29f8a32","created_at":"2026-09-14T09:30:05.629Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a43bd4e515330246393ac4233402a7d3","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.694Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-a3310d7cf4964ad6d29f8a32: start","job_status":"start","queue_duration_s":0.287043,"scheduling_latency_s":0.287056}1389gitlab # [ 152.916056] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"a3310d7cf4964ad6d29f8a32","created_at":"2026-09-14T09:30:05.629Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a43bd4e515330246393ac4233402a7d3","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.694Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-a3310d7cf4964ad6d29f8a32: done: 0.011406 sec","job_status":"done","queue_duration_s":0.287043,"scheduling_latency_s":0.287056,"redis_calls":10,"redis_duration_s":0.001316,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210921,"duration_s":0.011406,"completed_at":"2026-09-14T09:30:05.993Z","load_balancing_strategy":"primary","db_duration_s":0.000576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 152.976681] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:05.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"86f78f8b330b0e1dfad6928d","created_at":"2026-09-14T09:30:05.721Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d929147e311d13941de9c2ab56a02fba","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.736Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-86f78f8b330b0e1dfad6928d: start","job_status":"start","queue_duration_s":0.257327,"scheduling_latency_s":0.25734}1391gitlab # [ 152.977632] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.005Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"86f78f8b330b0e1dfad6928d","created_at":"2026-09-14T09:30:05.721Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d929147e311d13941de9c2ab56a02fba","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.736Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-86f78f8b330b0e1dfad6928d: done: 0.011619 sec","job_status":"done","queue_duration_s":0.257327,"scheduling_latency_s":0.25734,"redis_calls":11,"redis_duration_s":0.001622,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000185,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000233829,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.011619,"completed_at":"2026-09-14T09:30:06.005Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 153.017075] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1f41acbbe1f5cf5200af2e55","created_at":"2026-09-14T09:30:05.741Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7f1d3ba1d35169a6505d31cb53993dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.747Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1f41acbbe1f5cf5200af2e55: start","job_status":"start","queue_duration_s":0.258852,"scheduling_latency_s":0.258865}1393gitlab # [ 153.017732] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1f41acbbe1f5cf5200af2e55","created_at":"2026-09-14T09:30:05.741Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7f1d3ba1d35169a6505d31cb53993dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.747Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1f41acbbe1f5cf5200af2e55: done: 0.011016 sec","job_status":"done","queue_duration_s":0.258852,"scheduling_latency_s":0.258865,"redis_calls":10,"redis_duration_s":0.001252,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022908,"duration_s":0.011016,"completed_at":"2026-09-14T09:30:06.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 153.056629] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8a3cfdd9fa6125736a2db54a","created_at":"2026-09-14T09:30:05.758Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6e03b3050aed5c5bd2fb6d8aeb4fb4c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T09:30:05.770Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8a3cfdd9fa6125736a2db54a: start","job_status":"start","queue_duration_s":0.248245,"scheduling_latency_s":0.248258}1395gitlab # [ 153.065742] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.025Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1396gitlab # [ 153.068930] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"8a3cfdd9fa6125736a2db54a","created_at":"2026-09-14T09:30:05.758Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6e03b3050aed5c5bd2fb6d8aeb4fb4c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T09:30:05.770Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-8a3cfdd9fa6125736a2db54a: done: 0.010387 sec","job_status":"done","queue_duration_s":0.248245,"scheduling_latency_s":0.248258,"redis_calls":10,"redis_duration_s":0.001252,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210362,"duration_s":0.010387,"completed_at":"2026-09-14T09:30:06.029Z","load_balancing_strategy":"primary","db_duration_s":0.000246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 153.109080] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99153f93551255ed07564655","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99153f93551255ed07564655: start","job_status":"start","queue_duration_s":0.173016,"scheduling_latency_s":0.173029}1398gitlab # [ 153.148959] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99153f93551255ed07564655","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99153f93551255ed07564655: done: 0.012451 sec","job_status":"done","queue_duration_s":0.173016,"scheduling_latency_s":0.173029,"redis_calls":16,"redis_duration_s":0.0019950000000000002,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195835,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012451,"completed_at":"2026-09-14T09:30:06.042Z","load_balancing_strategy":"primary","db_duration_s":0.000237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 153.152259] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0d27a7597ddb6937440f2d4","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0d27a7597ddb6937440f2d4: start","job_status":"start","queue_duration_s":0.186333,"scheduling_latency_s":0.186347}1400gitlab # [ 153.163674] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0d27a7597ddb6937440f2d4","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0d27a7597ddb6937440f2d4: done: 0.009023 sec","job_status":"done","queue_duration_s":0.186333,"scheduling_latency_s":0.186347,"redis_calls":13,"redis_duration_s":0.0015739999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009023,"completed_at":"2026-09-14T09:30:06.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 153.211995] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c1f714e4f7547abca5d36ed","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c1f714e4f7547abca5d36ed: start","job_status":"start","queue_duration_s":0.196214,"scheduling_latency_s":0.196229}1402gitlab # [ 153.212750] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c1f714e4f7547abca5d36ed","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c1f714e4f7547abca5d36ed: done: 0.008935 sec","job_status":"done","queue_duration_s":0.196214,"scheduling_latency_s":0.196229,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008935,"completed_at":"2026-09-14T09:30:06.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 153.250984] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84447db0d2eae4ca12a38e2f","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84447db0d2eae4ca12a38e2f: start","job_status":"start","queue_duration_s":0.205964,"scheduling_latency_s":0.205977}1404gitlab # [ 153.254740] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84447db0d2eae4ca12a38e2f","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84447db0d2eae4ca12a38e2f: done: 0.008989 sec","job_status":"done","queue_duration_s":0.205964,"scheduling_latency_s":0.205977,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008989,"completed_at":"2026-09-14T09:30:06.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 153.297607] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3683aa3295d4c4705c3d4259","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3683aa3295d4c4705c3d4259: start","job_status":"start","queue_duration_s":0.218786,"scheduling_latency_s":0.218802}1406gitlab # [ 153.299700] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3683aa3295d4c4705c3d4259","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3683aa3295d4c4705c3d4259: done: 0.009667 sec","job_status":"done","queue_duration_s":0.218786,"scheduling_latency_s":0.218802,"redis_calls":13,"redis_duration_s":0.001922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009667,"completed_at":"2026-09-14T09:30:06.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 153.345334] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b28f3a75638ab51879771ffa","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b28f3a75638ab51879771ffa: start","job_status":"start","queue_duration_s":0.230045,"scheduling_latency_s":0.23006}1408gitlab # [ 153.346430] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b28f3a75638ab51879771ffa","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b28f3a75638ab51879771ffa: done: 0.016792 sec","job_status":"done","queue_duration_s":0.230045,"scheduling_latency_s":0.23006,"redis_calls":13,"redis_duration_s":0.0017720000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016792,"completed_at":"2026-09-14T09:30:06.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 153.407982] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91c78c97c9f3e511e3dbaacb","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91c78c97c9f3e511e3dbaacb: start","job_status":"start","queue_duration_s":0.248013,"scheduling_latency_s":0.248029}1410gitlab # [ 153.408701] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91c78c97c9f3e511e3dbaacb","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91c78c97c9f3e511e3dbaacb: done: 0.031554 sec","job_status":"done","queue_duration_s":0.248013,"scheduling_latency_s":0.248029,"redis_calls":13,"redis_duration_s":0.014539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.013925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031554,"completed_at":"2026-09-14T09:30:06.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 153.460908] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57c51e96aa5163c61749c82a","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57c51e96aa5163c61749c82a: start","job_status":"start","queue_duration_s":0.280892,"scheduling_latency_s":0.280914}1412gitlab # [ 153.512314] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57c51e96aa5163c61749c82a","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57c51e96aa5163c61749c82a: done: 0.657079 sec","job_status":"done","queue_duration_s":0.280892,"scheduling_latency_s":0.280914,"redis_calls":13,"redis_duration_s":0.010168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00295,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.657079,"completed_at":"2026-09-14T09:30:06.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 153.550496] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"118a92bd4ad94c060c8b8df5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-118a92bd4ad94c060c8b8df5: start","job_status":"start","queue_duration_s":0.975972,"scheduling_latency_s":0.976028}1414gitlab # [ 153.589248] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"118a92bd4ad94c060c8b8df5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-118a92bd4ad94c060c8b8df5: done: 0.038754 sec","job_status":"done","queue_duration_s":0.975972,"scheduling_latency_s":0.976028,"redis_calls":13,"redis_duration_s":0.013571999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003302,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038754,"completed_at":"2026-09-14T09:30:06.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 153.628905] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b57438744be31e662818ccdf","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b57438744be31e662818ccdf: start","job_status":"start","queue_duration_s":1.054545,"scheduling_latency_s":1.054591}1416gitlab # [ 153.711374] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b57438744be31e662818ccdf","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b57438744be31e662818ccdf: done: 0.045195 sec","job_status":"done","queue_duration_s":1.054545,"scheduling_latency_s":1.054591,"redis_calls":13,"redis_duration_s":0.020419,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005133,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.015286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.045195,"completed_at":"2026-09-14T09:30:06.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 153.755824] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:06.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e35936201b16c15666969c2e","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e35936201b16c15666969c2e: start","job_status":"start","queue_duration_s":1.138629,"scheduling_latency_s":1.138655}1418gitlab # [ 153.756821] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e35936201b16c15666969c2e","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e35936201b16c15666969c2e: done: 0.012714 sec","job_status":"done","queue_duration_s":1.138629,"scheduling_latency_s":1.138655,"redis_calls":13,"redis_duration_s":0.0028380000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012714,"completed_at":"2026-09-14T09:30:07.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 153.810401] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a35117894510b891fe5df751","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a35117894510b891fe5df751: start","job_status":"start","queue_duration_s":1.152462,"scheduling_latency_s":1.152477}1420gitlab # [ 153.811417] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a35117894510b891fe5df751","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a35117894510b891fe5df751: done: 0.010453 sec","job_status":"done","queue_duration_s":1.152462,"scheduling_latency_s":1.152477,"redis_calls":13,"redis_duration_s":0.002699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010453,"completed_at":"2026-09-14T09:30:07.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 153.857545] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1482d15151910f3f8d4e268","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1482d15151910f3f8d4e268: start","job_status":"start","queue_duration_s":1.163997,"scheduling_latency_s":1.164011}1422gitlab # [ 153.858473] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1482d15151910f3f8d4e268","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1482d15151910f3f8d4e268: done: 0.010627 sec","job_status":"done","queue_duration_s":1.163997,"scheduling_latency_s":1.164011,"redis_calls":13,"redis_duration_s":0.002695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001763,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010627,"completed_at":"2026-09-14T09:30:07.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 153.898459] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb6915ff6285c758714540c9","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb6915ff6285c758714540c9: start","job_status":"start","queue_duration_s":1.176074,"scheduling_latency_s":1.176089}1424gitlab # [ 153.899308] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb6915ff6285c758714540c9","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb6915ff6285c758714540c9: done: 0.010026 sec","job_status":"done","queue_duration_s":1.176074,"scheduling_latency_s":1.176089,"redis_calls":13,"redis_duration_s":0.002051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010026,"completed_at":"2026-09-14T09:30:07.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 153.943075] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f589a54f6496ec0c0aba569","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f589a54f6496ec0c0aba569: start","job_status":"start","queue_duration_s":1.186974,"scheduling_latency_s":1.186987}1426gitlab # [ 153.943959] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f589a54f6496ec0c0aba569","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f589a54f6496ec0c0aba569: done: 0.009417 sec","job_status":"done","queue_duration_s":1.186974,"scheduling_latency_s":1.186987,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009417,"completed_at":"2026-09-14T09:30:07.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 153.968747] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8374e6223feac9bf6d6667a","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8374e6223feac9bf6d6667a: start","job_status":"start","queue_duration_s":1.197469,"scheduling_latency_s":1.19749}1428gitlab # [ 154.008837] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8374e6223feac9bf6d6667a","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8374e6223feac9bf6d6667a: done: 0.010695 sec","job_status":"done","queue_duration_s":1.197469,"scheduling_latency_s":1.19749,"redis_calls":13,"redis_duration_s":0.002713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010695,"completed_at":"2026-09-14T09:30:07.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 154.015806] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa2f1b3e55563526ad1bd2ff","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa2f1b3e55563526ad1bd2ff: start","job_status":"start","queue_duration_s":1.209249,"scheduling_latency_s":1.209263}1430gitlab # [ 154.064784] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa2f1b3e55563526ad1bd2ff","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa2f1b3e55563526ad1bd2ff: done: 0.010928 sec","job_status":"done","queue_duration_s":1.209249,"scheduling_latency_s":1.209263,"redis_calls":13,"redis_duration_s":0.00264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010928,"completed_at":"2026-09-14T09:30:07.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 154.065918] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03f375103ca5b6505633fb08","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03f375103ca5b6505633fb08: start","job_status":"start","queue_duration_s":1.221297,"scheduling_latency_s":1.221312}1432gitlab # [ 154.111735] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03f375103ca5b6505633fb08","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03f375103ca5b6505633fb08: done: 0.010459 sec","job_status":"done","queue_duration_s":1.221297,"scheduling_latency_s":1.221312,"redis_calls":13,"redis_duration_s":0.00301,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010459,"completed_at":"2026-09-14T09:30:07.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 154.112922] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"405576e4c06cd9e8ecf8a4b8","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-405576e4c06cd9e8ecf8a4b8: start","job_status":"start","queue_duration_s":1.232882,"scheduling_latency_s":1.232897}1434gitlab # [ 154.159920] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"405576e4c06cd9e8ecf8a4b8","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-405576e4c06cd9e8ecf8a4b8: done: 0.009535 sec","job_status":"done","queue_duration_s":1.232882,"scheduling_latency_s":1.232897,"redis_calls":13,"redis_duration_s":0.001691,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009535,"completed_at":"2026-09-14T09:30:07.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 154.160904] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8d359ec4159085be1263c7d","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8d359ec4159085be1263c7d: start","job_status":"start","queue_duration_s":1.243274,"scheduling_latency_s":1.243287}1436gitlab # [ 154.206798] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8d359ec4159085be1263c7d","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8d359ec4159085be1263c7d: done: 0.009336 sec","job_status":"done","queue_duration_s":1.243274,"scheduling_latency_s":1.243287,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009336,"completed_at":"2026-09-14T09:30:07.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 154.207961] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9052271c467096781787eac1","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9052271c467096781787eac1: start","job_status":"start","queue_duration_s":1.254934,"scheduling_latency_s":1.254989}1438gitlab # [ 154.218549] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9052271c467096781787eac1","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9052271c467096781787eac1: done: 0.023865 sec","job_status":"done","queue_duration_s":1.254934,"scheduling_latency_s":1.254989,"redis_calls":13,"redis_duration_s":0.003169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023865,"completed_at":"2026-09-14T09:30:07.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 154.278771] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3750b7c6f8701831e73eb9e1","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3750b7c6f8701831e73eb9e1: start","job_status":"start","queue_duration_s":1.280274,"scheduling_latency_s":1.2803}1440gitlab # [ 154.279575] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3750b7c6f8701831e73eb9e1","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3750b7c6f8701831e73eb9e1: done: 0.015561 sec","job_status":"done","queue_duration_s":1.280274,"scheduling_latency_s":1.2803,"redis_calls":13,"redis_duration_s":0.002524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015561,"completed_at":"2026-09-14T09:30:07.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 154.320644] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bee68b4c890a3ca44cfe00f5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bee68b4c890a3ca44cfe00f5: start","job_status":"start","queue_duration_s":1.296845,"scheduling_latency_s":1.296865}1442gitlab # [ 154.321630] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bee68b4c890a3ca44cfe00f5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bee68b4c890a3ca44cfe00f5: done: 0.011188 sec","job_status":"done","queue_duration_s":1.296845,"scheduling_latency_s":1.296865,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011188,"completed_at":"2026-09-14T09:30:07.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 154.370958] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27ea5edc9fa2db03ade611a6","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27ea5edc9fa2db03ade611a6: start","job_status":"start","queue_duration_s":1.309145,"scheduling_latency_s":1.30916}1444gitlab # [ 154.371613] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27ea5edc9fa2db03ade611a6","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27ea5edc9fa2db03ade611a6: done: 0.010182 sec","job_status":"done","queue_duration_s":1.309145,"scheduling_latency_s":1.30916,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010182,"completed_at":"2026-09-14T09:30:07.176Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 154.413689] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c25753180421eef5ab074cc","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c25753180421eef5ab074cc: start","job_status":"start","queue_duration_s":1.320136,"scheduling_latency_s":1.320151}1446gitlab # [ 154.414620] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c25753180421eef5ab074cc","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c25753180421eef5ab074cc: done: 0.009496 sec","job_status":"done","queue_duration_s":1.320136,"scheduling_latency_s":1.320151,"redis_calls":13,"redis_duration_s":0.0015799999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009496,"completed_at":"2026-09-14T09:30:07.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 154.459715] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61545c0f5b7a36261ccf2415","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61545c0f5b7a36261ccf2415: start","job_status":"start","queue_duration_s":1.330574,"scheduling_latency_s":1.330588}1448gitlab # [ 154.460500] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61545c0f5b7a36261ccf2415","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61545c0f5b7a36261ccf2415: done: 0.009512 sec","job_status":"done","queue_duration_s":1.330574,"scheduling_latency_s":1.330588,"redis_calls":13,"redis_duration_s":0.001664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009512,"completed_at":"2026-09-14T09:30:07.196Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 154.482744] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5f65fecbc099c7ff428187f","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5f65fecbc099c7ff428187f: start","job_status":"start","queue_duration_s":1.340943,"scheduling_latency_s":1.340958}1450gitlab # [ 154.494519] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5f65fecbc099c7ff428187f","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5f65fecbc099c7ff428187f: done: 0.009971 sec","job_status":"done","queue_duration_s":1.340943,"scheduling_latency_s":1.340958,"redis_calls":13,"redis_duration_s":0.001621,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009971,"completed_at":"2026-09-14T09:30:07.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 154.541631] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6af200a0388fb4727b394de","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6af200a0388fb4727b394de: start","job_status":"start","queue_duration_s":1.351776,"scheduling_latency_s":1.351791}1452gitlab # [ 154.542615] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6af200a0388fb4727b394de","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6af200a0388fb4727b394de: done: 0.009521 sec","job_status":"done","queue_duration_s":1.351776,"scheduling_latency_s":1.351791,"redis_calls":13,"redis_duration_s":0.001566,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009521,"completed_at":"2026-09-14T09:30:07.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 154.597634] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2456f4d4d409431e99a1a333","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2456f4d4d409431e99a1a333: start","job_status":"start","queue_duration_s":1.362116,"scheduling_latency_s":1.36213}1454gitlab # [ 154.598626] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2456f4d4d409431e99a1a333","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2456f4d4d409431e99a1a333: done: 0.009556 sec","job_status":"done","queue_duration_s":1.362116,"scheduling_latency_s":1.36213,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009556,"completed_at":"2026-09-14T09:30:07.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 154.620489] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12e27eda16574c521e8bd066","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12e27eda16574c521e8bd066: start","job_status":"start","queue_duration_s":1.372943,"scheduling_latency_s":1.372958}1456gitlab # [ 154.661210] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12e27eda16574c521e8bd066","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12e27eda16574c521e8bd066: done: 0.009498 sec","job_status":"done","queue_duration_s":1.372943,"scheduling_latency_s":1.372958,"redis_calls":13,"redis_duration_s":0.001707,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009498,"completed_at":"2026-09-14T09:30:07.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 154.701249] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500a5cc45eb5caee4bbe5e4a","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500a5cc45eb5caee4bbe5e4a: start","job_status":"start","queue_duration_s":1.383304,"scheduling_latency_s":1.383317}1458gitlab # [ 154.702038] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500a5cc45eb5caee4bbe5e4a","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500a5cc45eb5caee4bbe5e4a: done: 0.009543 sec","job_status":"done","queue_duration_s":1.383304,"scheduling_latency_s":1.383317,"redis_calls":13,"redis_duration_s":0.001648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009543,"completed_at":"2026-09-14T09:30:07.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 154.710248] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"581ed05823e62686543f7b86","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-581ed05823e62686543f7b86: start","job_status":"start","queue_duration_s":1.393718,"scheduling_latency_s":1.393732}1460gitlab # [ 154.721747] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"581ed05823e62686543f7b86","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-581ed05823e62686543f7b86: done: 0.009422 sec","job_status":"done","queue_duration_s":1.393718,"scheduling_latency_s":1.393732,"redis_calls":13,"redis_duration_s":0.001609,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009422,"completed_at":"2026-09-14T09:30:07.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 154.774986] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b67cb4b33761c770c1621a0","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b67cb4b33761c770c1621a0: start","job_status":"start","queue_duration_s":1.404379,"scheduling_latency_s":1.404393}1462gitlab # [ 154.775689] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b67cb4b33761c770c1621a0","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b67cb4b33761c770c1621a0: done: 0.009924 sec","job_status":"done","queue_duration_s":1.404379,"scheduling_latency_s":1.404393,"redis_calls":13,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001478,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009924,"completed_at":"2026-09-14T09:30:07.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 154.824579] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8001d9a89bf6c2757389be10","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8001d9a89bf6c2757389be10: start","job_status":"start","queue_duration_s":1.415126,"scheduling_latency_s":1.41514}1464gitlab # [ 154.825473] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8001d9a89bf6c2757389be10","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8001d9a89bf6c2757389be10: done: 0.00957 sec","job_status":"done","queue_duration_s":1.415126,"scheduling_latency_s":1.41514,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00957,"completed_at":"2026-09-14T09:30:07.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 154.848755] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fad155cdaa3c726aebf51b42","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fad155cdaa3c726aebf51b42: start","job_status":"start","queue_duration_s":1.426402,"scheduling_latency_s":1.426449}1466gitlab # [ 154.860612] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fad155cdaa3c726aebf51b42","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fad155cdaa3c726aebf51b42: done: 0.025802 sec","job_status":"done","queue_duration_s":1.426402,"scheduling_latency_s":1.426449,"redis_calls":13,"redis_duration_s":0.008438000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025802,"completed_at":"2026-09-14T09:30:07.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 154.910033] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74eb1080c9ef109212b345f3","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74eb1080c9ef109212b345f3: start","job_status":"start","queue_duration_s":1.453648,"scheduling_latency_s":1.454095}1468gitlab # [ 154.910782] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74eb1080c9ef109212b345f3","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74eb1080c9ef109212b345f3: done: 0.01518 sec","job_status":"done","queue_duration_s":1.453648,"scheduling_latency_s":1.454095,"redis_calls":13,"redis_duration_s":0.0026509999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001634,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01518,"completed_at":"2026-09-14T09:30:07.325Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 154.951394] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e7b8baced908ba142e8d72d","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e7b8baced908ba142e8d72d: start","job_status":"start","queue_duration_s":1.469688,"scheduling_latency_s":1.469702}1470gitlab # [ 154.952169] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e7b8baced908ba142e8d72d","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e7b8baced908ba142e8d72d: done: 0.009172 sec","job_status":"done","queue_duration_s":1.469688,"scheduling_latency_s":1.469702,"redis_calls":13,"redis_duration_s":0.001724,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009172,"completed_at":"2026-09-14T09:30:07.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 155.007711] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a245033acf6f901b8df487b","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a245033acf6f901b8df487b: start","job_status":"start","queue_duration_s":1.479705,"scheduling_latency_s":1.479718}1472gitlab # [ 155.008571] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a245033acf6f901b8df487b","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a245033acf6f901b8df487b: done: 0.009249 sec","job_status":"done","queue_duration_s":1.479705,"scheduling_latency_s":1.479718,"redis_calls":13,"redis_duration_s":0.00166,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009249,"completed_at":"2026-09-14T09:30:07.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 155.032589] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92e82afa1e2143577e1bcc70","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92e82afa1e2143577e1bcc70: start","job_status":"start","queue_duration_s":1.489811,"scheduling_latency_s":1.489825}1474gitlab # [ 155.043765] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92e82afa1e2143577e1bcc70","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92e82afa1e2143577e1bcc70: done: 0.013876 sec","job_status":"done","queue_duration_s":1.489811,"scheduling_latency_s":1.489825,"redis_calls":13,"redis_duration_s":0.002755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013876,"completed_at":"2026-09-14T09:30:07.360Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 155.100523] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6911801d662b1ebe0785a9d5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6911801d662b1ebe0785a9d5: start","job_status":"start","queue_duration_s":1.504967,"scheduling_latency_s":1.504982}1476gitlab # [ 155.101300] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6911801d662b1ebe0785a9d5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6911801d662b1ebe0785a9d5: done: 0.009761 sec","job_status":"done","queue_duration_s":1.504967,"scheduling_latency_s":1.504982,"redis_calls":13,"redis_duration_s":0.001979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009761,"completed_at":"2026-09-14T09:30:07.371Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 155.144199] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15c96c11cd147ca405c6a2e5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15c96c11cd147ca405c6a2e5: start","job_status":"start","queue_duration_s":1.515588,"scheduling_latency_s":1.515601}1478gitlab # [ 155.144905] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15c96c11cd147ca405c6a2e5","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15c96c11cd147ca405c6a2e5: done: 0.009758 sec","job_status":"done","queue_duration_s":1.515588,"scheduling_latency_s":1.515601,"redis_calls":13,"redis_duration_s":0.001885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009758,"completed_at":"2026-09-14T09:30:07.382Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 155.187700] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beed0d9921253bf1b96e28cb","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beed0d9921253bf1b96e28cb: start","job_status":"start","queue_duration_s":1.526201,"scheduling_latency_s":1.526214}1480gitlab # [ 155.188545] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beed0d9921253bf1b96e28cb","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beed0d9921253bf1b96e28cb: done: 0.009404 sec","job_status":"done","queue_duration_s":1.526201,"scheduling_latency_s":1.526214,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009404,"completed_at":"2026-09-14T09:30:07.392Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 155.216507] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4109388935e3156100c0310","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4109388935e3156100c0310: start","job_status":"start","queue_duration_s":1.536431,"scheduling_latency_s":1.536445}1482gitlab # [ 155.259741] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4109388935e3156100c0310","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4109388935e3156100c0310: done: 0.014526 sec","job_status":"done","queue_duration_s":1.536431,"scheduling_latency_s":1.536445,"redis_calls":13,"redis_duration_s":0.003296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001615,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001681,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014526,"completed_at":"2026-09-14T09:30:07.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 155.261894] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"785ad45d50adfe0fbe548900","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-785ad45d50adfe0fbe548900: start","job_status":"start","queue_duration_s":1.552237,"scheduling_latency_s":1.552251}1484gitlab # [ 155.302383] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"785ad45d50adfe0fbe548900","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-785ad45d50adfe0fbe548900: done: 0.009286 sec","job_status":"done","queue_duration_s":1.552237,"scheduling_latency_s":1.552251,"redis_calls":13,"redis_duration_s":0.0016870000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009286,"completed_at":"2026-09-14T09:30:07.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 155.307153] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8bd5859a74e9c073cd25c44","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8bd5859a74e9c073cd25c44: start","job_status":"start","queue_duration_s":1.562361,"scheduling_latency_s":1.562375}1486gitlab # [ 155.346685] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8bd5859a74e9c073cd25c44","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8bd5859a74e9c073cd25c44: done: 0.00935 sec","job_status":"done","queue_duration_s":1.562361,"scheduling_latency_s":1.562375,"redis_calls":13,"redis_duration_s":0.001719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00935,"completed_at":"2026-09-14T09:30:07.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 155.386642] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"334030bcc4eaaac7eca475ac","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-334030bcc4eaaac7eca475ac: start","job_status":"start","queue_duration_s":1.572562,"scheduling_latency_s":1.572576}1488gitlab # [ 155.386985] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"334030bcc4eaaac7eca475ac","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-334030bcc4eaaac7eca475ac: done: 0.009342 sec","job_status":"done","queue_duration_s":1.572562,"scheduling_latency_s":1.572576,"redis_calls":13,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009342,"completed_at":"2026-09-14T09:30:07.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 155.428238] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf3e32a88ad7876705d4408","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf3e32a88ad7876705d4408: start","job_status":"start","queue_duration_s":1.582892,"scheduling_latency_s":1.582906}1490gitlab # [ 155.429037] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acf3e32a88ad7876705d4408","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acf3e32a88ad7876705d4408: done: 0.015023 sec","job_status":"done","queue_duration_s":1.582892,"scheduling_latency_s":1.582906,"redis_calls":13,"redis_duration_s":0.006576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015023,"completed_at":"2026-09-14T09:30:07.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 155.443620] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f555381b5aaddd039a2176e","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f555381b5aaddd039a2176e: start","job_status":"start","queue_duration_s":1.598805,"scheduling_latency_s":1.59882}1492gitlab # [ 155.455159] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f555381b5aaddd039a2176e","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f555381b5aaddd039a2176e: done: 0.009288 sec","job_status":"done","queue_duration_s":1.598805,"scheduling_latency_s":1.59882,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009288,"completed_at":"2026-09-14T09:30:07.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 155.502188] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7d30d7cc424b3d0d38e3c63","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7d30d7cc424b3d0d38e3c63: start","job_status":"start","queue_duration_s":1.609031,"scheduling_latency_s":1.609044}1494gitlab # [ 155.502961] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7d30d7cc424b3d0d38e3c63","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7d30d7cc424b3d0d38e3c63: done: 0.009279 sec","job_status":"done","queue_duration_s":1.609031,"scheduling_latency_s":1.609044,"redis_calls":13,"redis_duration_s":0.001705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009279,"completed_at":"2026-09-14T09:30:07.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 155.544753] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a55b1260eb6beb73aca87c7","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a55b1260eb6beb73aca87c7: start","job_status":"start","queue_duration_s":1.619138,"scheduling_latency_s":1.61915}1496gitlab # [ 155.545774] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a55b1260eb6beb73aca87c7","created_at":"2026-09-14T09:30:05.810Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-da80da775c5a4a93","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"43ae6a6ed327260f5da86d6f4234342b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:05.856Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a55b1260eb6beb73aca87c7: done: 0.008983 sec","job_status":"done","queue_duration_s":1.619138,"scheduling_latency_s":1.61915,"redis_calls":13,"redis_duration_s":0.001624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008983,"completed_at":"2026-09-14T09:30:07.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 155.590123] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.485Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"e7c8acedeb42a2c41f13b42d","created_at":"2026-09-14T09:30:06.094Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef58a234833026573997a8544d2b31dc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.103Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-e7c8acedeb42a2c41f13b42d: start","job_status":"start","queue_duration_s":1.382901,"scheduling_latency_s":1.382914}1498gitlab # [ 155.591986] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.514Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"e7c8acedeb42a2c41f13b42d","created_at":"2026-09-14T09:30:06.094Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef58a234833026573997a8544d2b31dc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.103Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-e7c8acedeb42a2c41f13b42d: done: 0.029027 sec","job_status":"done","queue_duration_s":1.382901,"scheduling_latency_s":1.382914,"redis_calls":10,"redis_duration_s":0.010223,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001627,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0009856,"duration_s":0.029027,"completed_at":"2026-09-14T09:30:07.514Z","load_balancing_strategy":"primary","db_duration_s":0.005745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 155.651372] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b58a148c50977200800b09cf","created_at":"2026-09-14T09:30:06.113Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47eac2111662345f91cb9d96cc06d849","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.135Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b58a148c50977200800b09cf: start","job_status":"start","queue_duration_s":1.383007,"scheduling_latency_s":1.383022}1500gitlab # [ 155.652127] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b58a148c50977200800b09cf","created_at":"2026-09-14T09:30:06.113Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47eac2111662345f91cb9d96cc06d849","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.135Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b58a148c50977200800b09cf: done: 0.025595 sec","job_status":"done","queue_duration_s":1.383007,"scheduling_latency_s":1.383022,"redis_calls":12,"redis_duration_s":0.002267,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000380216,"exclusive_lock_hold_duration_s":0.008111646,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.025595,"completed_at":"2026-09-14T09:30:07.543Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 155.711581] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.544Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7958b053e638fc111f3edfe0","created_at":"2026-09-14T09:30:06.762Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"494c308182279297b4bb148e4c7d8a27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T09:30:06.777Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7958b053e638fc111f3edfe0: start","job_status":"start","queue_duration_s":0.766837,"scheduling_latency_s":0.766851}1502gitlab # [ 155.712526] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7958b053e638fc111f3edfe0","created_at":"2026-09-14T09:30:06.762Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"494c308182279297b4bb148e4c7d8a27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T09:30:06.777Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7958b053e638fc111f3edfe0: done: 0.017677 sec","job_status":"done","queue_duration_s":0.766837,"scheduling_latency_s":0.766851,"redis_calls":10,"redis_duration_s":0.001414,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000216228,"duration_s":0.017677,"completed_at":"2026-09-14T09:30:07.562Z","load_balancing_strategy":"primary","db_duration_s":0.002196,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 155.729251] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"145a9d3eb7c4f9ee47672f93","created_at":"2026-09-14T09:30:06.785Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24b63f1ef18f85afeb82d7e28be95fc6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.793Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-145a9d3eb7c4f9ee47672f93: start","job_status":"start","queue_duration_s":0.770597,"scheduling_latency_s":0.770612}1504gitlab # [ 155.769760] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"145a9d3eb7c4f9ee47672f93","created_at":"2026-09-14T09:30:06.785Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"24b63f1ef18f85afeb82d7e28be95fc6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.793Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-145a9d3eb7c4f9ee47672f93: done: 0.014624 sec","job_status":"done","queue_duration_s":0.770597,"scheduling_latency_s":0.770612,"redis_calls":12,"redis_duration_s":0.001921,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000314565,"exclusive_lock_hold_duration_s":0.0034208249999999997,"duration_s":0.014624,"completed_at":"2026-09-14T09:30:07.579Z","load_balancing_strategy":"primary","db_duration_s":0.001632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 155.812621] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"4776a7946ad46110df1db068","created_at":"2026-09-14T09:30:06.836Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28511f7bc8a8b0c3ab4aad8bccb443d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.848Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-4776a7946ad46110df1db068: start","job_status":"start","queue_duration_s":0.731009,"scheduling_latency_s":0.731023}1506gitlab # [ 155.813313] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"4776a7946ad46110df1db068","created_at":"2026-09-14T09:30:06.836Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28511f7bc8a8b0c3ab4aad8bccb443d3","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.848Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-4776a7946ad46110df1db068: done: 0.011137 sec","job_status":"done","queue_duration_s":0.731009,"scheduling_latency_s":0.731023,"redis_calls":12,"redis_duration_s":0.001619,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214273,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.011137,"completed_at":"2026-09-14T09:30:07.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 155.857814] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"d7a864f39789b06a6bfa13dd","created_at":"2026-09-14T09:30:06.856Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17562dbb8ef455c204385441a34802a4","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.866Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-d7a864f39789b06a6bfa13dd: start","job_status":"start","queue_duration_s":0.725224,"scheduling_latency_s":0.725238}1508gitlab # [ 155.858656] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Gitlab::Import::ImportFileCleanupWorker","jid":"d7a864f39789b06a6bfa13dd","created_at":"2026-09-14T09:30:06.856Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17562dbb8ef455c204385441a34802a4","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:872203d04eabf30611038084c617b8ff07fd3e5607b1d8e2c977d615a3cfd3cb","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.866Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Gitlab::Import::ImportFileCleanupWorker JID-d7a864f39789b06a6bfa13dd: done: 0.019989 sec","job_status":"done","queue_duration_s":0.725224,"scheduling_latency_s":0.725238,"redis_calls":10,"redis_duration_s":0.006792,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000293,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000932242,"duration_s":0.019989,"completed_at":"2026-09-14T09:30:07.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 155.906919] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"53f935804d68d5f8cdeaf64c","created_at":"2026-09-14T09:30:06.908Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0de01ca8dd6ca004518f4b91b327347e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.925Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-53f935804d68d5f8cdeaf64c: start","job_status":"start","queue_duration_s":0.690072,"scheduling_latency_s":0.690087}1510gitlab # [ 155.907794] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:07.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"53f935804d68d5f8cdeaf64c","created_at":"2026-09-14T09:30:06.908Z","trace_propagation_headers":{"sentry-trace":"3375e06e603a46e587589d4b1576e770-dbfc99163e754d2b","baggage":"sentry-trace_id=3375e06e603a46e587589d4b1576e770,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0de01ca8dd6ca004518f4b91b327347e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:06.925Z","job_size_bytes":2,"pid":1620,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-53f935804d68d5f8cdeaf64c: done: 0.014985 sec","job_status":"done","queue_duration_s":0.690072,"scheduling_latency_s":0.690087,"redis_calls":10,"redis_duration_s":0.001445,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208966,"duration_s":0.014985,"completed_at":"2026-09-14T09:30:07.630Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 156.118226] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1705168443 wd_nsec: 17051681641512gitlab # [ 160.281735] setup-gitlab-start[2220]: W, [2026-09-14T09:30:13.563831 #2220] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1513gitlab # [ 180.801848] postgres[2414]: [2414] ERROR: relation "table_without_models" does not exist1514gitlab # [ 180.802501] postgres[2414]: [2414] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1515gitlab # [ 183.566767] runuser[2219]: pam_unix(runuser:session): session closed for user gitlab1516gitlab # [ 183.569867] setup-gitlab-start[2197]: + systemctl restart gitlab-sidekiq.service1517gitlab # [ 183.593614] systemd[1]: Stopping gitlab-sidekiq.service...1518gitlab # [ 183.594228] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:36.876Z","message":"Shutting down"}1519gitlab # [ 183.598541] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:36.881Z","message":"Scheduler exiting..."}1520gitlab # [ 183.599152] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:36.881Z","message":"Terminating quiet threads for default capsule"}1521gitlab # [ 183.599815] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:36.882Z","message":"Scheduler exiting..."}1522gitlab # [ 184.100983] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:37.383Z","message":"Pausing to allow jobs to finish..."}1523gitlab # [ 185.105724] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:38.387Z","message":"Bye!"}1524gitlab # [ 185.127382] sidekiq-cluster[1620]: {"severity":"INFO","time":"2026-09-14T09:30:38.410Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1620,"worker_id":"sidekiq_0","memwd_rss_bytes":944943104,"retry":0}1525gitlab # [ 186.385870] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1526gitlab # [ 186.388444] systemd[1]: Stopped gitlab-sidekiq.service.1527gitlab # [ 186.390392] systemd[1]: gitlab-sidekiq.service: Consumed 56.621s CPU time over 1min 53.133s wall clock time, 957.1M memory peak, 4.4M read from disk.1528gitlab # [ 186.394935] systemd[1]: Started gitlab-sidekiq.service.1529gitlab # [ 186.397379] setup-gitlab-start[2197]: + mkdir -p /var/lib/secrets1530gitlab # [ 186.399632] setup-gitlab-start[2197]: + echo nixbot-test-token-of-20-chars1531gitlab # [ 186.401968] setup-gitlab-start[2197]: + chmod 644 /var/lib/secrets/gitlab-token1532gitlab # [ 186.404508] setup-gitlab-start[2197]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1533gitlab # [ 186.824353] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:40 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 3911534gitlab # [ 187.408469] gitaly[905]: time="2026-09-14T09:30:40.690Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:50.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.690" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1535gitlab # [ 187.426315] gitaly[905]: time="2026-09-14T09:30:40.690Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:50.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.690" grpc.time_ms=0.74 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1536gitlab # [ 187.575533] gitaly[905]: time="2026-09-14T09:30:40.858Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:50.177" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.857" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 187.597785] gitaly[905]: time="2026-09-14T09:30:40.858Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:50.177" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.857" grpc.time_ms=0.476 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1538gitlab # [ 187.673786] gitaly[905]: time="2026-09-14T09:30:40.956Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK diskcache=76b78d2c-da46-4318-8505-34010463d150 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T09:30:50.206" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.904" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1539gitlab # [ 187.694169] gitaly[905]: time="2026-09-14T09:30:40.956Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1582 command.oublock=32 command.real_time_ms=24 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T09:30:50.206" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.904" grpc.time_ms=52.195 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1540gitlab # [ 187.711657] gitaly[905]: time="2026-09-14T09:30:40.973Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:50.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.972" pid=905 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1541gitlab # [ 187.721180] gitaly[905]: time="2026-09-14T09:30:40.973Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:50.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:40.972" grpc.time_ms=0.716 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1542gitlab # [ 187.748276] gitaly[905]: time="2026-09-14T09:30:41.031Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:30:51.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.020" grpc.time_ms=10.753 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1543gitlab # [ 187.815149] gitaly[905]: time="2026-09-14T09:30:41.097Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:51.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.097" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1544gitlab # [ 187.815481] gitaly[905]: time="2026-09-14T09:30:41.098Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:51.087" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.097" grpc.time_ms=0.287 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1545gitlab # [ 187.902745] gitaly[905]: time="2026-09-14T09:30:41.185Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK diskcache=42e2f7a4-ef1f-49fb-86f2-19d8523a922a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T09:30:51.127" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.137" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1546gitlab # [ 187.920471] gitaly[905]: time="2026-09-14T09:30:41.185Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1579 command.oublock=32 command.real_time_ms=22 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T09:30:51.127" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.137" grpc.time_ms=47.81 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 187.944977] gitaly[905]: time="2026-09-14T09:30:41.226Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:51.215" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.225" pid=905 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1548gitlab # [ 187.945760] gitaly[905]: time="2026-09-14T09:30:41.226Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:51.215" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.225" grpc.time_ms=0.735 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1549gitlab # [ 187.964588] gitaly[905]: time="2026-09-14T09:30:41.247Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:30:51.223" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.235" grpc.time_ms=11.924 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1550gitlab # [ 187.994683] gitaly[905]: time="2026-09-14T09:30:41.277Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:30:51.267" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.277" grpc.time_ms=0.132 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1551gitlab # [ 187.999452] gitaly[905]: time="2026-09-14T09:30:41.282Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T09:30:51.271" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.281" grpc.time_ms=0.233 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1552gitlab # [ 188.420968] gitaly[905]: time="2026-09-14T09:30:41.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM27KYJGHNHMH6TJV7SEYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:30:51.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:41.690" grpc.time_ms=12.053 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1553gitlab # [ 188.786224] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:42 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 19411554gitlab # [ 188.790745] setup-gitlab-start[2460]: {"id":1,"message":null,"error":null}1555gitlab # [ 189.460664] sidekiq-cluster[2448]: {"severity":"INFO","time":"2026-09-14T09:30:42.726Z","message":"Starting cluster with 1 processes"}1556gitlab # [ 193.927115] gitaly[905]: time="2026-09-14T09:30:47.209Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EDR0CVADDK7A3YX82KM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:30:57.186" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:47.197" grpc.time_ms=11.454 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1557gitlab # [ 194.006356] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:47 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1881558gitlab # [ 194.013458] setup-gitlab-start[2197]: + mkdir -p /tmp/test-flake1559gitlab # [ 194.015656] setup-gitlab-start[2197]: + cd /tmp/test-flake1560gitlab # [ 194.017794] setup-gitlab-start[2197]: + git init -b master1561gitlab # [ 194.035978] setup-gitlab-start[2534]: Initialized empty Git repository in /tmp/test-flake/.git/1562gitlab # [ 194.039071] setup-gitlab-start[2197]: + git config user.name test1563gitlab # [ 194.041184] setup-gitlab-start[2197]: + git config user.email test@example.com1564gitlab # [ 194.043553] setup-gitlab-start[2197]: + cat1565gitlab # [ 194.045292] setup-gitlab-start[2197]: + git add flake.nix1566gitlab # [ 194.046595] setup-gitlab-start[2197]: + git commit -m 'initial commit'1567gitlab # [ 194.065645] setup-gitlab-start[2539]: [master (root-commit) 30a78e6] initial commit1568gitlab # [ 194.066346] setup-gitlab-start[2539]: 1 file changed, 10 insertions(+)1569gitlab # [ 194.068133] setup-gitlab-start[2539]: create mode 100644 flake.nix1570gitlab # [ 194.070220] setup-gitlab-start[2197]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1571gitlab # [ 194.072297] setup-gitlab-start[2197]: + git push -u origin master1572gitlab # [ 194.137498] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:47 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 401573gitlab # [ 194.225601] gitaly[905]: time="2026-09-14T09:30:47.508Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EQXMBKS79B7TH75PXMK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:47.496" grpc.time_ms=11.889 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1574gitlab # [ 194.255493] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:47 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1161575gitlab # [ 194.530213] gitaly[905]: time="2026-09-14T09:30:47.812Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:30:57.120" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:47.801" grpc.time_ms=11.416 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1576gitlab # [ 194.570654] gitaly[905]: time="2026-09-14T09:30:47.853Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-14T09:31:17.038" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-14T09:30:47.838" grpc.time_ms=14.599 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1577gitlab # [ 194.617340] gitaly[905]: time="2026-09-14T09:30:47.899Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 duration_ms=182ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/allowed"1578gitlab # [ 194.790763] gitaly[905]: time="2026-09-14T09:30:48.073Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 duration_ms=170ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/pre_receive"1579gitlab # [ 194.797717] gitaly[905]: time="2026-09-14T09:30:48.080Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5311 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:47.716" grpc.time_ms=363.956 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1580gitlab # [ 194.858539] gitaly[905]: time="2026-09-14T09:30:48.141Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:48.140" grpc.time_ms=1.062 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1581gitlab # [ 194.923258] gitaly[905]: time="2026-09-14T09:30:48.206Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:48.203" grpc.time_ms=2.41 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1582gitlab # [ 194.979344] gitaly[905]: time="2026-09-14T09:30:48.261Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:48.260" grpc.time_ms=0.661 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1583gitlab # [ 195.111843] gitaly[905]: time="2026-09-14T09:30:48.394Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:30:58.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:48.385" grpc.time_ms=9.212 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 195.197780] gitaly[905]: time="2026-09-14T09:30:48.480Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1565 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-14T09:30:58.080" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:30:48.461" grpc.time_ms=18.938 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 195.239365] gitaly[905]: time="2026-09-14T09:30:48.522Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 duration_ms=224ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/post_receive"1586gitlab # [ 195.246705] gitaly[905]: time="2026-09-14T09:30:48.522Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4805 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:48.297" grpc.time_ms=225.503 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1587gitlab # [ 195.270277] gitaly[905]: time="2026-09-14T09:30:48.553Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 diskcache=5972f8f6-84b6-4423-897b-1aba9856f67d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:47.626" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1588gitlab # [ 195.280067] gitlab-workhorse[832]: time="2026-09-14T09:30:48Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521589gitlab # [ 195.280587] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:48 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 10061590gitlab # [ 195.281091] gitaly[905]: time="2026-09-14T09:30:48.553Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=273 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13909 command.oublock=32 command.real_time_ms=916 command.spawn_token_fork_ms=1 command.system_time_ms=187 command.user_time_ms=86 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:47.626" grpc.time_ms=926.316 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1591gitlab # [ 195.297604] setup-gitlab-start[2543]: To http://gitlab/root/test-flake.git1592gitlab # [ 195.297932] setup-gitlab-start[2543]: * [new branch] master -> master1593gitlab # [ 195.301482] setup-gitlab-start[2543]: branch 'master' set up to track 'origin/master'.1594gitlab # [ 195.305144] systemd[1]: Finished setup-gitlab.service.1595gitlab # [ 195.307725] systemd[1]: Starting nixbot CI service...1596gitlab: (finished: waiting for unit setup-gitlab.service, in 196.91 seconds)1597gitlab: waiting for unit nixbot.service1598gitlab # [ 196.468472] nixbot[2634]: {"ts": "2026-09-14T09:30:49.750Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1599gitlab # [ 196.505683] nixbot[2634]: {"ts": "2026-09-14T09:30:49.788Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1600gitlab # [ 196.526444] nixbot[2634]: {"ts": "2026-09-14T09:30:49.809Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1601gitlab # [ 196.532247] nixbot[2634]: {"ts": "2026-09-14T09:30:49.814Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1602gitlab # [ 196.538102] nixbot[2634]: {"ts": "2026-09-14T09:30:49.820Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1603gitlab # [ 196.544405] nixbot[2634]: {"ts": "2026-09-14T09:30:49.827Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1604gitlab # [ 196.549482] nixbot[2634]: {"ts": "2026-09-14T09:30:49.832Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1605gitlab # [ 196.553345] nixbot[2634]: {"ts": "2026-09-14T09:30:49.836Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1606gitlab # [ 196.559753] nixbot[2634]: {"ts": "2026-09-14T09:30:49.842Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1607gitlab # [ 196.563693] nixbot[2634]: {"ts": "2026-09-14T09:30:49.846Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1608gitlab # [ 196.573262] nixbot[2634]: {"ts": "2026-09-14T09:30:49.855Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1609gitlab # [ 196.580074] nixbot[2634]: {"ts": "2026-09-14T09:30:49.862Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1610gitlab # [ 196.584789] nixbot[2634]: {"ts": "2026-09-14T09:30:49.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1611gitlab # [ 196.588886] nixbot[2634]: {"ts": "2026-09-14T09:30:49.871Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1612gitlab # [ 196.596582] nixbot[2634]: {"ts": "2026-09-14T09:30:49.879Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1613gitlab # [ 196.600681] nixbot[2634]: {"ts": "2026-09-14T09:30:49.883Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1614gitlab # [ 196.604664] nixbot[2634]: {"ts": "2026-09-14T09:30:49.887Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1615gitlab # [ 196.608561] nixbot[2634]: {"ts": "2026-09-14T09:30:49.891Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1616gitlab # [ 196.616555] nixbot[2634]: {"ts": "2026-09-14T09:30:49.899Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1617gitlab # [ 196.620407] nixbot[2634]: {"ts": "2026-09-14T09:30:49.903Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1618gitlab # [ 196.624487] nixbot[2634]: {"ts": "2026-09-14T09:30:49.907Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1619gitlab # [ 196.630378] nixbot[2634]: {"ts": "2026-09-14T09:30:49.913Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1620gitlab # [ 196.635916] nixbot[2634]: {"ts": "2026-09-14T09:30:49.918Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1621gitlab # [ 196.640392] nixbot[2634]: {"ts": "2026-09-14T09:30:49.923Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1622gitlab # [ 196.644389] nixbot[2634]: {"ts": "2026-09-14T09:30:49.927Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1623gitlab # [ 196.648925] nixbot[2634]: {"ts": "2026-09-14T09:30:49.931Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1624gitlab # [ 196.653117] nixbot[2634]: {"ts": "2026-09-14T09:30:49.935Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1625gitlab # [ 196.656946] nixbot[2634]: {"ts": "2026-09-14T09:30:49.939Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1626gitlab # [ 196.660874] nixbot[2634]: {"ts": "2026-09-14T09:30:49.943Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1627gitlab # [ 196.679770] nixbot[2634]: {"ts": "2026-09-14T09:30:49.962Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1628gitlab # [ 196.685899] nixbot[2634]: {"ts": "2026-09-14T09:30:49.968Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1629gitlab # [ 196.689979] nixbot[2634]: {"ts": "2026-09-14T09:30:49.972Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1630gitlab # [ 196.693975] nixbot[2634]: {"ts": "2026-09-14T09:30:49.976Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1631gitlab # [ 196.699421] nixbot[2634]: {"ts": "2026-09-14T09:30:49.982Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1632gitlab # [ 196.703408] nixbot[2634]: {"ts": "2026-09-14T09:30:49.986Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1633gitlab # [ 196.709880] nixbot[2634]: {"ts": "2026-09-14T09:30:49.992Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1634gitlab # [ 196.714607] nixbot[2634]: {"ts": "2026-09-14T09:30:49.997Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1635gitlab # [ 197.037192] nixbot[2634]: INFO: Started server process [2634]1636gitlab # [ 197.038104] nixbot[2634]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1637gitlab # [ 197.055527] nixbot[2634]: INFO: ::ffff:127.0.0.1:53246 - "GET /health HTTP/1.1" 200 OK1638gitlab # [ 197.067676] systemd[1]: Started nixbot CI service.1639gitlab # [ 197.070600] systemd[1]: Reached target Multi-User System.1640gitlab # [ 197.071569] systemd[1]: Startup finished in 1.135s (kernel) + 3.620s (initrd) + 3min 12.316s (userspace) = 3min 17.071s.1641gitlab # [ 197.274170] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:50 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2191642gitlab # [ 197.275833] nixbot[2634]: {"ts": "2026-09-14T09:30:50.558Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1643gitlab # [ 197.292201] nixbot[2634]: {"ts": "2026-09-14T09:30:50.574Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1644gitlab: (finished: waiting for unit nixbot.service, in 1.05 seconds)1645gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1646gitlab # [ 197.358149] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 541647gitlab # [ 197.360957] nixbot[2634]: {"ts": "2026-09-14T09:30:50.643Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1648gitlab # [ 197.364192] nixbot[2634]: {"ts": "2026-09-14T09:30:50.647Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1649gitlab # [ 197.370510] nixbot[2634]: INFO: ::ffff:127.0.0.1:53260 - "GET /health HTTP/1.1" 200 OK1650gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1651subtest: project discovered and webhook registered1652gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1653gitlab # [ 197.439684] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 381654gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds)1655gitlab # [ 197.593435] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 2241656gitlab # [ 197.596819] nixbot[2634]: {"ts": "2026-09-14T09:30:50.878Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1657gitlab # [ 198.143830] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 5391658gitlab # [ 198.146238] nixbot[2634]: {"ts": "2026-09-14T09:30:51.428Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1659gitlab # [ 198.194589] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 361660gitlab # [ 198.197558] nixbot[2634]: {"ts": "2026-09-14T09:30:51.480Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1661gitlab # [ 198.243703] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 391662gitlab # [ 198.245963] nixbot[2634]: {"ts": "2026-09-14T09:30:51.525Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1663gitlab # [ 198.297791] gitaly[905]: time="2026-09-14T09:30:51.580Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2JRCDJK6CKFXH6N4XG3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-14T09:31:21.064" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:30:51.564" grpc.time_ms=14.965 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 198.320363] gitaly[905]: time="2026-09-14T09:30:51.603Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2JRCDJK6CKFXH6N4XG3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:31:21.101" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:30:51.602" grpc.time_ms=0.688 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1665gitlab # [ 198.344157] gitaly[905]: time="2026-09-14T09:30:51.627Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2JRCDJK6CKFXH6N4XG3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-14T09:31:01.235" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:30:51.615" grpc.time_ms=11.352 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1666gitlab # [ 198.382237] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1311667gitlab # [ 198.384592] nixbot[2634]: {"ts": "2026-09-14T09:30:51.665Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1668gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1669gitlab # [ 198.471326] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 791670gitlab # [ 198.472766] nixbot[2634]: {"ts": "2026-09-14T09:30:51.755Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1671gitlab # [ 198.477409] nixbot[2634]: {"ts": "2026-09-14T09:30:51.760Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1672gitlab # [ 198.559596] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 651673gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds)1674(finished: subtest: project discovered and webhook registered, in 1.19 seconds)1675subtest: push delivers the webhook1676gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1677gitlab # [ 198.564953] gitaly[905]: time="2026-09-14T09:30:51.847Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2K0SWV8V0E0RZDX45BY2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.836" grpc.time_ms=11.546 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1678gitlab # [ 198.570871] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 521679gitlab # [ 198.642932] gitaly[905]: time="2026-09-14T09:30:51.925Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2K2VEPH3052VDNCH1EKT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.918" pid=905 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1680gitlab # [ 198.643231] gitaly[905]: time="2026-09-14T09:30:51.926Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2K2VEPH3052VDNCH1EKT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.918" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1681gitlab # [ 198.643971] gitaly[905]: time="2026-09-14T09:30:51.926Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2K2VEPH3052VDNCH1EKT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.918" grpc.time_ms=8.169 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1682gitlab # [ 198.646963] gitlab-workhorse[832]: time="2026-09-14T09:30:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131683gitlab # [ 198.650885] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 591684gitlab # [ 198.720856] gitaly[905]: time="2026-09-14T09:30:52.003Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2K4XRQPC1VQEFQYSK8MX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.992" pid=905 remote_ip="2001:db8:1::1" request_sha=b36382ec11190051ec5a8f948db5a62e4a91b629 response_bytes=223 span.kind=server system=grpc1685gitlab # [ 198.721401] gitaly[905]: time="2026-09-14T09:30:52.003Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1204 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2K4XRQPC1VQEFQYSK8MX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.992" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1686gitlab # [ 198.722030] gitaly[905]: time="2026-09-14T09:30:52.004Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1204 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2K4XRQPC1VQEFQYSK8MX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:51.992" grpc.time_ms=11.166 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1687gitlab # [ 198.755292] gitlab-workhorse[832]: time="2026-09-14T09:30:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231688gitlab # [ 198.757347] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1051689gitlab # [ 198.844175] gitaly[905]: time="2026-09-14T09:30:52.126Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2K96XXB1GBWPDBSRE661 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.115" grpc.time_ms=11.203 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1690gitlab # [ 198.868603] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 801691gitlab # [ 198.934267] gitaly[905]: time="2026-09-14T09:30:52.217Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KBV4SQBBEKBSFRQ8EQH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.203" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1692gitlab # [ 198.935190] gitaly[905]: time="2026-09-14T09:30:52.217Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KBV4SQBBEKBSFRQ8EQH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.203" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1693gitlab # [ 198.936609] gitaly[905]: time="2026-09-14T09:30:52.217Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KBV4SQBBEKBSFRQ8EQH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.203" grpc.time_ms=14.165 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1694gitlab # [ 198.991417] gitlab-workhorse[832]: time="2026-09-14T09:30:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671695gitlab # [ 198.992140] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1181696gitlab # [ 199.099767] gitaly[905]: time="2026-09-14T09:30:52.382Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KGMYM0EDYP26YR6EFT1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.366" grpc.time_ms=15.714 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1697gitlab # [ 199.101546] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 751698gitlab # [ 199.165386] gitaly[905]: time="2026-09-14T09:30:52.448Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KK17NS2CB8HJM334WHS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.429" pid=905 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1699gitlab # [ 199.165652] gitaly[905]: time="2026-09-14T09:30:52.448Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KK17NS2CB8HJM334WHS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.429" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1700gitlab # [ 199.166870] gitaly[905]: time="2026-09-14T09:30:52.449Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KK17NS2CB8HJM334WHS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:30:52.429" grpc.time_ms=19.48 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1701gitlab # [ 199.167765] gitlab-workhorse[832]: time="2026-09-14T09:30:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091702gitlab # [ 199.170382] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 641703gitlab # [ 199.382086] gitaly[905]: time="2026-09-14T09:30:52.664Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:31:22.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:30:52.662" grpc.time_ms=1.435 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1704gitlab # [ 199.461679] gitaly[905]: time="2026-09-14T09:30:52.744Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-14T09:31:22.130" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:30:52.731" grpc.time_ms=12.988 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705gitlab # [ 199.488385] gitaly[905]: time="2026-09-14T09:30:52.771Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:02.080" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:30:52.760" grpc.time_ms=10.318 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706gitlab # [ 199.524466] gitaly[905]: time="2026-09-14T09:30:52.807Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:02.119" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:30:52.800" grpc.time_ms=7.167 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1707gitlab # [ 199.583202] gitaly[905]: time="2026-09-14T09:30:52.866Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T09:31:02.184" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:52.865" grpc.time_ms=0.386 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1708gitlab # [ 199.625073] gitaly[905]: time="2026-09-14T09:30:52.907Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=8 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T09:31:02.215" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:30:52.895" grpc.time_ms=11.633 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1709gitlab # [ 199.680994] gitaly[905]: time="2026-09-14T09:30:52.963Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:52.963" grpc.time_ms=0.377 pid=905 span.kind=server system=grpc1710gitlab # [ 199.713592] gitaly[905]: time="2026-09-14T09:30:52.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:30:52.996" grpc.time_ms=0.322 pid=905 span.kind=server system=grpc1711gitlab # [ 199.731346] gitaly[905]: time="2026-09-14T09:30:53.014Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V diskcache=d1a89fb5-3137-4ec2-ba84-f1f6fe6285e8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T09:31:02.244" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:52.924" pid=905 ref=refs/keep-around/30a78e656f1740d433631748372c3a791991c03c remote_ip="2001:db8:1::1" revision=30a78e656f1740d433631748372c3a791991c03c span.kind=server system=grpc username=root1712gitlab # [ 199.731721] gitaly[905]: time="2026-09-14T09:30:53.014Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=60 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5245 command.oublock=8 command.real_time_ms=80 command.spawn_token_fork_ms=1 command.system_time_ms=33 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T09:31:02.244" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:30:52.924" grpc.time_ms=89.583 pid=905 ref=refs/keep-around/30a78e656f1740d433631748372c3a791991c03c remote_ip="2001:db8:1::1" revision=30a78e656f1740d433631748372c3a791991c03c span.kind=server system=grpc username=root1713gitlab # [ 199.736042] gitaly[905]: time="2026-09-14T09:30:53.018Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:31:23.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:30:53.018" grpc.time_ms=0.269 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714gitlab # [ 199.873152] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:30:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6181715gitlab # [ 199.873963] nixbot[2634]: {"ts": "2026-09-14T09:30:53.156Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1716gitlab # [ 199.881574] nixbot[2634]: {"ts": "2026-09-14T09:30:53.164Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-30a78e656f17-0"}1717gitlab # [ 200.042994] nixbot[2634]: {"ts": "2026-09-14T09:30:53.325Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-30a78e656f17-0", "argv0": "bwrap"}1718gitlab # [ 200.157427] systemd[1]: Started Nix Daemon.1719gitlab # [ 200.199942] nix-daemon[2793]: accepted connection from pid 2788, user nixbot1720gitlab # [ 204.256899] nix-daemon[2793]: accepted connection from pid 2835, user nixbot1721gitlab # [ 204.273479] nix-daemon[2793]: accepted connection from pid 2788, user nixbot1722gitlab # [ 208.834468] nix-daemon[2793]: accepted connection from pid 2788, user nixbot1723gitlab # [ 208.931057] nix-daemon[2793]: accepted connection from pid 2877, user nixbot1724gitlab # [ 208.971417] gitaly[905]: time="2026-09-14T09:31:02.253Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2X5MBK2ECVXS678YHVZK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:31:32.051" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:02.252" grpc.time_ms=0.937 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725gitlab # [ 209.092191] gitaly[905]: time="2026-09-14T09:31:02.374Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2X5MBK2ECVXS678YHVZK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:12.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:02.365" grpc.time_ms=9.719 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1726gitlab # [ 209.201638] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2861727gitlab # [ 209.209798] nixbot[2634]: {"ts": "2026-09-14T09:31:02.490Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1728gitlab # [ 209.262206] gitaly[905]: time="2026-09-14T09:31:02.545Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2XF93K0WYP1FHBD44WN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:31:32.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:02.543" grpc.time_ms=0.752 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729gitlab # [ 209.292341] gitaly[905]: time="2026-09-14T09:31:02.575Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2XF93K0WYP1FHBD44WN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:12.185" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:02.565" grpc.time_ms=9.278 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730gitlab # [ 209.330780] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1071731gitlab # [ 209.331713] nixbot[2634]: {"ts": "2026-09-14T09:31:02.614Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1732gitlab # [ 213.168633] nix-daemon[2793]: accepted connection from pid 2939, user nixbot1733gitlab # [ 213.242758] gitaly[905]: time="2026-09-14T09:31:06.524Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM31BND3HKT1DEPE2KZHM8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:31:36.523" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:06.523" grpc.time_ms=0.974 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1734gitlab # [ 213.291809] gitaly[905]: time="2026-09-14T09:31:06.574Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM31BND3HKT1DEPE2KZHM8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:16.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:06.566" grpc.time_ms=7.873 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1735gitlab # [ 213.329961] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:06 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1261736gitlab # [ 213.333098] nixbot[2634]: {"ts": "2026-09-14T09:31:06.615Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1737gitlab # [ 213.719235] gitaly[905]: time="2026-09-14T09:31:07.001Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM31SHHKHDNT562M48421A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:06.988" grpc.time_ms=12.749 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1738gitlab # [ 213.748556] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1011739gitlab # [ 213.804076] gitaly[905]: time="2026-09-14T09:31:07.086Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM31WX9X0EXT7B92X2ZJPB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.074" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1740gitlab # [ 213.804967] gitaly[905]: time="2026-09-14T09:31:07.086Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM31WX9X0EXT7B92X2ZJPB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.074" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1741gitlab # [ 213.806470] gitlab-workhorse[832]: time="2026-09-14T09:31:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671742gitlab # [ 213.807517] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 501743gitlab # [ 213.809104] gitaly[905]: time="2026-09-14T09:31:07.087Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM31WX9X0EXT7B92X2ZJPB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.074" grpc.time_ms=12.427 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1744gitlab # [ 213.942727] gitaly[905]: time="2026-09-14T09:31:07.225Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3211VP4AN56RK8YQ8JVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.217" grpc.time_ms=7.606 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1745gitlab # [ 213.966332] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 791746gitlab # [ 214.022507] gitaly[905]: time="2026-09-14T09:31:07.305Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM323JV9NSGZ7VBR55G8B8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.295" pid=905 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1747gitlab # [ 214.023559] gitaly[905]: time="2026-09-14T09:31:07.305Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM323JV9NSGZ7VBR55G8B8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.295" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1748gitlab # [ 214.024595] gitlab-workhorse[832]: time="2026-09-14T09:31:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671749gitlab # [ 214.025617] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 551750gitlab # [ 214.027185] gitaly[905]: time="2026-09-14T09:31:07.306Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM323JV9NSGZ7VBR55G8B8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:07.295" grpc.time_ms=10.063 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1751gitlab # [ 214.482781] sidekiq-cluster[2506]: 2026-09-14T09:31:07.764Z pid=2506 tid=xuq INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2506, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>449572864}1752gitlab # [ 214.651179] sidekiq-cluster[2506]: W, [2026-09-14T09:31:07.933872 #2506] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1753gitlab # [ 214.688185] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:07.970Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1754gitlab # [ 216.685198] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:09.967Z","message":"GitLab reliable fetch activated!"}1755gitlab # [ 216.688894] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:09.967Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1756gitlab # [ 235.431619] postgres[3183]: [3183] ERROR: relation "table_without_models" does not exist1757gitlab # [ 235.432442] postgres[3183]: [3183] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1758gitlab # [ 237.838197] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:31.120Z","message":"Booted Rails 7.2.3.1 application in production environment"}1759gitlab # [ 237.838983] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:31.120Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1760gitlab # [ 237.841112] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:31.120Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1761gitlab # [ 237.843145] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:31.120Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1762gitlab # [ 237.844486] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:31.121Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1763gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 39.55 seconds)1764gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1765gitlab # [ 238.216221] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:31 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 331766gitlab # [ 238.327218] gitaly[905]: time="2026-09-14T09:31:31.609Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SSF3YZ5V03306DCRQ99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:31.595" grpc.time_ms=13.365 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1767gitlab # [ 238.327703] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:31 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 1051768gitlab # Enumerating objects: 5, done.1769gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1770gitlab # Delta compression using up to 4 threads1771gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1772gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1773gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1774gitlab # [ 238.573409] gitaly[905]: time="2026-09-14T09:31:31.856Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-14T09:32:01.848" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-14T09:31:31.848" grpc.time_ms=7.086 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 238.603943] gitaly[905]: time="2026-09-14T09:31:31.886Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 duration_ms=82ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/allowed"1776gitlab # [ 238.627076] gitaly[905]: time="2026-09-14T09:31:31.909Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 duration_ms=22ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/pre_receive"1777gitlab # [ 238.633100] gitaly[905]: time="2026-09-14T09:31:31.916Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5311 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:31.803" grpc.time_ms=112.805 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1778gitlab # [ 238.695600] gitaly[905]: time="2026-09-14T09:31:31.978Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:31.977" grpc.time_ms=0.746 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1779gitlab # [ 238.747616] gitaly[905]: time="2026-09-14T09:31:32.030Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:32.029" grpc.time_ms=0.568 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1780gitlab # [ 238.784218] gitaly[905]: time="2026-09-14T09:31:32.067Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:32.066" grpc.time_ms=0.435 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1781gitlab # [ 238.851753] gitaly[905]: time="2026-09-14T09:31:32.134Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 duration_ms=35ns method=POST pid=905 status=200 url="http://unix/api/v4/internal/post_receive"1782gitlab # [ 238.851995] gitaly[905]: time="2026-09-14T09:31:32.134Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4805 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:32.098" grpc.time_ms=36.479 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1783gitlab # [ 238.876977] gitaly[905]: time="2026-09-14T09:31:32.159Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 diskcache=b1ef2ebb-3d94-461f-8105-3b954bbf0a17 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:31.719" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1784gitlab # [ 238.886447] gitaly[905]: time="2026-09-14T09:31:32.169Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=239 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13476 command.oublock=32 command.real_time_ms=429 command.spawn_token_fork_ms=0 command.system_time_ms=144 command.user_time_ms=95 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:31.719" grpc.time_ms=440.174 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1785gitlab # [ 238.899210] gitlab-workhorse[832]: time="2026-09-14T09:31:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521786gitlab # [ 238.899697] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:32 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 5151787gitlab # To http://gitlab/root/test-flake.git1788gitlab # 30a78e6..a4867ee master -> master1789gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.79 seconds)1790gitlab: must succeed: git -C /tmp/test-flake rev-parse master1791gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1792gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1793gitlab # [ 238.992504] nixbot[2634]: INFO: ::ffff:127.0.0.1:42316 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1794gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.08 seconds)1795gitlab # [ 239.104963] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:32.386Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"75e2f1562e355bc6a5e26105","created_at":"2026-09-14T09:30:41.046Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.080Z","job_size_bytes":2,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-75e2f1562e355bc6a5e26105: start","job_status":"start","queue_duration_s":51.305988,"scheduling_latency_s":51.306031}1796gitlab # [ 239.347335] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:32.629Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"75e2f1562e355bc6a5e26105","created_at":"2026-09-14T09:30:41.046Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.080Z","job_size_bytes":2,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-75e2f1562e355bc6a5e26105: done: 0.242729 sec","job_status":"done","queue_duration_s":51.305988,"scheduling_latency_s":51.306031,"redis_calls":25,"redis_duration_s":0.00849,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004672,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002143,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001675,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.105681867,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.242729,"completed_at":"2026-09-14T09:31:32.629Z","load_balancing_strategy":"primary","db_duration_s":0.067928,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1797gitlab # [ 239.405731] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:32.687Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"42530d9b5fcbace5233387c0","created_at":"2026-09-14T09:30:41.089Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.093Z","job_size_bytes":2,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-42530d9b5fcbace5233387c0: start","job_status":"start","queue_duration_s":51.594269,"scheduling_latency_s":51.594298}1798gitlab # [ 239.526405] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:32.808Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"42530d9b5fcbace5233387c0","created_at":"2026-09-14T09:30:41.089Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.093Z","job_size_bytes":2,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-42530d9b5fcbace5233387c0: done: 0.121129 sec","job_status":"done","queue_duration_s":51.594269,"scheduling_latency_s":51.594298,"redis_calls":14,"redis_duration_s":0.004344,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001147,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001803,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082097306,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.121129,"completed_at":"2026-09-14T09:31:32.808Z","load_balancing_strategy":"primary","db_duration_s":0.025124,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1799gitlab # [ 239.601491] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:32.883Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"a4d2dfdb6e840148109aa462","created_at":"2026-09-14T09:30:41.618Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.623Z","job_size_bytes":5,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-a4d2dfdb6e840148109aa462: start","job_status":"start","queue_duration_s":51.260678,"scheduling_latency_s":51.260708}1800gitlab # [ 239.785759] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.068Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"a4d2dfdb6e840148109aa462","created_at":"2026-09-14T09:30:41.618Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.623Z","job_size_bytes":5,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-a4d2dfdb6e840148109aa462: done: 0.184304 sec","job_status":"done","queue_duration_s":51.260678,"scheduling_latency_s":51.260708,"redis_calls":12,"redis_duration_s":0.004193,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001133,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002433,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000324622,"duration_s":0.184304,"completed_at":"2026-09-14T09:31:33.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029526,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1801gitlab # [ 239.864650] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.147Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"67bf7451a58d3d08d1ea1373","created_at":"2026-09-14T09:30:41.644Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.647Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-67bf7451a58d3d08d1ea1373: start","job_status":"start","queue_duration_s":51.499384,"scheduling_latency_s":51.49943}1802gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1803gitlab # [ 240.036460] nixbot[2634]: INFO: ::ffff:127.0.0.1:42328 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1804gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.04 seconds)1805gitlab # [ 240.104063] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.385Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"67bf7451a58d3d08d1ea1373","created_at":"2026-09-14T09:30:41.644Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.647Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-67bf7451a58d3d08d1ea1373: done: 0.239129 sec","job_status":"done","queue_duration_s":51.499384,"scheduling_latency_s":51.49943,"redis_calls":11,"redis_duration_s":0.003139,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000684,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001816,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.176,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.176,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000494197,"duration_s":0.239129,"completed_at":"2026-09-14T09:31:33.385Z","load_balancing_strategy":"primary","db_duration_s":0.079327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1806gitlab # [ 240.178883] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.460Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9a6b5ecf50b34b9e598e0b74","created_at":"2026-09-14T09:30:41.649Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.652Z","job_size_bytes":89,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9a6b5ecf50b34b9e598e0b74: start","job_status":"start","queue_duration_s":51.807335,"scheduling_latency_s":51.807385}1807gitlab # [ 240.231798] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.514Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9a6b5ecf50b34b9e598e0b74","created_at":"2026-09-14T09:30:41.649Z","trace_propagation_headers":{"sentry-trace":"a379846c8ad44a6390ef382329dcc1ca-b38a892547d74df7","baggage":"sentry-trace_id=a379846c8ad44a6390ef382329dcc1ca,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM27KYJGHNHMH6TJV7SEYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:41.652Z","job_size_bytes":89,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9a6b5ecf50b34b9e598e0b74: done: 0.054102 sec","job_status":"done","queue_duration_s":51.807335,"scheduling_latency_s":51.807385,"redis_calls":11,"redis_duration_s":0.003657,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001079,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000496152,"duration_s":0.054102,"completed_at":"2026-09-14T09:31:33.513Z","load_balancing_strategy":"primary","db_duration_s":0.007178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1808gitlab # [ 240.285197] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.567Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 30a78e656f1740d433631748372c3a791991c03c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5b45c85129dc83b76dbb1df1","created_at":"2026-09-14T09:30:48.331Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-0b3028bd474c46c5","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:48.351Z","job_size_bytes":185,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5b45c85129dc83b76dbb1df1: start","job_status":"start","queue_duration_s":45.21552,"scheduling_latency_s":45.215582}1809gitlab # [ 240.312943] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:33.595Z","message":"changes: \"0000000000000000000000000000000000000000 30a78e656f1740d433631748372c3a791991c03c refs/heads/master\\n\""}1810gitlab # [ 240.368797] gitaly[905]: time="2026-09-14T09:31:33.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:43.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:33.638" grpc.time_ms=11.893 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 240.413333] gitaly[905]: time="2026-09-14T09:31:33.696Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:31:43.053" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:33.684" grpc.time_ms=11.684 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 240.512621] gitaly[905]: time="2026-09-14T09:31:33.795Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-14T09:31:43.149" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:33.779" grpc.time_ms=15.34 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 240.575140] gitaly[905]: time="2026-09-14T09:31:33.857Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3932 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:33.857" grpc.time_ms=0.655 pid=905 span.kind=server system=grpc1814gitlab # [ 240.634700] gitaly[905]: time="2026-09-14T09:31:33.917Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3934 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:33.916" grpc.time_ms=0.623 pid=905 span.kind=server system=grpc1815gitlab # [ 240.667642] gitaly[905]: time="2026-09-14T09:31:33.950Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 diskcache=b60efc67-fc9b-426b-84b2-07e9178d7957 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T09:31:43.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:33.812" pid=905 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1816gitlab # [ 240.668438] gitaly[905]: time="2026-09-14T09:31:33.950Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=139 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5221 command.oublock=8 command.real_time_ms=126 command.spawn_token_fork_ms=0 command.system_time_ms=96 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T09:31:43.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:33.812" grpc.time_ms=137.915 pid=905 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1817gitlab # [ 240.692376] gitaly[905]: time="2026-09-14T09:31:33.975Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T09:31:43.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:33.974" grpc.time_ms=0.245 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 240.712380] gitaly[905]: time="2026-09-14T09:31:33.995Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=945 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-14T09:31:43.010" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:33.980" grpc.time_ms=14.701 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab # [ 240.794663] gitaly[905]: time="2026-09-14T09:31:34.077Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-14T09:32:04.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:34.070" grpc.time_ms=6.857 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab # [ 240.849921] gitaly[905]: time="2026-09-14T09:31:34.132Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T09:31:44.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:34.131" grpc.time_ms=0.478 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 240.872431] gitaly[905]: time="2026-09-14T09:31:34.142Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-14T09:31:44.004" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-14T09:31:34.134" grpc.time_ms=7.718 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 240.891464] gitaly[905]: time="2026-09-14T09:31:34.174Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:44.033" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:34.163" grpc.time_ms=10.472 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1823gitlab # [ 240.920503] gitaly[905]: time="2026-09-14T09:31:34.203Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:44.065" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:34.195" grpc.time_ms=7.591 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 240.947912] gitaly[905]: time="2026-09-14T09:31:34.230Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=958 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-14T09:32:04.213" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:34.214" grpc.time_ms=16.467 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1826gitlab # [ 241.084509] nixbot[2634]: INFO: ::ffff:127.0.0.1:42340 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1827gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.05 seconds)1828gitlab # [ 241.408715] gitaly[905]: time="2026-09-14T09:31:34.690Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-14T09:31:44.058" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:34.689" grpc.time_ms=1.575 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829gitlab # [ 241.436995] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:34.717Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 30a78e656f1740d433631748372c3a791991c03c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5b45c85129dc83b76dbb1df1","created_at":"2026-09-14T09:30:48.331Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-0b3028bd474c46c5","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:30:48.351Z","job_size_bytes":185,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5b45c85129dc83b76dbb1df1: done: 1.150927 sec","job_status":"done","queue_duration_s":45.21552,"scheduling_latency_s":45.215582,"gitaly_calls":13,"gitaly_duration_s":0.375612,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.074636,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.004585,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.00812,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002009,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00503,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.042974,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011918,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.203,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.224,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.670846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00102527,"exclusive_lock_hold_duration_s":0.066080187,"duration_s":1.150927,"completed_at":"2026-09-14T09:31:34.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.103376,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1830gitlab # [ 241.498822] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:34.721Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"29a023bf1f3d513272de2f3c","created_at":"2026-09-14T09:30:53.022Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.027Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-29a023bf1f3d513272de2f3c: start","job_status":"start","queue_duration_s":41.693339,"scheduling_latency_s":41.693393}1831gitlab # [ 241.530708] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:34.813Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"29a023bf1f3d513272de2f3c","created_at":"2026-09-14T09:30:53.022Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.027Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-29a023bf1f3d513272de2f3c: done: 0.092476 sec","job_status":"done","queue_duration_s":41.693339,"scheduling_latency_s":41.693393,"redis_calls":16,"redis_duration_s":0.004462,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00226,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345854,"duration_s":0.092476,"completed_at":"2026-09-14T09:31:34.813Z","load_balancing_strategy":"primary","db_duration_s":0.021911,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1832gitlab # [ 241.552457] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:34.814Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a1419a2605b16fa409cb96b3","created_at":"2026-09-14T09:30:53.028Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.031Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a1419a2605b16fa409cb96b3: start","job_status":"start","queue_duration_s":41.783396,"scheduling_latency_s":41.783411}1833gitlab # [ 241.607440] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:34.889Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a1419a2605b16fa409cb96b3","created_at":"2026-09-14T09:30:53.028Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.031Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a1419a2605b16fa409cb96b3: done: 0.075241 sec","job_status":"done","queue_duration_s":41.783396,"scheduling_latency_s":41.783411,"redis_calls":21,"redis_duration_s":0.004877,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002121,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284673,"duration_s":0.075241,"completed_at":"2026-09-14T09:31:34.889Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1834gitlab # [ 241.609112] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:34.891Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"08251a0c8a09944296132d7d","created_at":"2026-09-14T09:30:53.116Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.128Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-08251a0c8a09944296132d7d: start","job_status":"start","queue_duration_s":41.762678,"scheduling_latency_s":41.762693}1835gitlab # [ 241.798301] gitaly[905]: time="2026-09-14T09:31:35.081Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:35.080" grpc.time_ms=0.397 pid=905 span.kind=server system=grpc1836gitlab # [ 241.852639] gitaly[905]: time="2026-09-14T09:31:35.135Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:35.134" grpc.time_ms=0.658 pid=905 span.kind=server system=grpc1837gitlab # [ 241.911450] gitaly[905]: time="2026-09-14T09:31:35.194Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:35.193" grpc.time_ms=0.925 pid=905 span.kind=server system=grpc1838gitlab # [ 241.939342] gitaly[905]: time="2026-09-14T09:31:35.222Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V diskcache=18e0f5d8-d0d0-4e1d-b372-e5b6c9459f51 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:05.039" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:35.039" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 241.948643] gitaly[905]: time="2026-09-14T09:31:35.231Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=142 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7589 command.oublock=0 command.real_time_ms=173 command.spawn_token_fork_ms=0 command.system_time_ms=87 command.user_time_ms=55 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:05.039" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:35.039" grpc.time_ms=182.388 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1841gitlab # [ 242.077144] gitaly[905]: time="2026-09-14T09:31:35.359Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:05.357" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:35.358" grpc.time_ms=1.278 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 242.107560] gitaly[905]: time="2026-09-14T09:31:35.390Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2KQRVPT0Z697A4EQRY7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:05.387" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:35.388" grpc.time_ms=1.832 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 242.136505] nixbot[2634]: INFO: ::ffff:127.0.0.1:42344 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1844gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.05 seconds)1845gitlab # [ 242.153325] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.435Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"08251a0c8a09944296132d7d","created_at":"2026-09-14T09:30:53.116Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.128Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2506,"job_status":"rescheduled","message":"PipelineProcessWorker JID-08251a0c8a09944296132d7d: rescheduled"}1846gitlab # [ 242.165765] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.448Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"08251a0c8a09944296132d7d","created_at":"2026-09-14T09:30:53.116Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-492281dc60444529","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:30:53.128Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-08251a0c8a09944296132d7d: done: 0.556701 sec","job_status":"done","queue_duration_s":41.762678,"scheduling_latency_s":41.762693,"gitaly_calls":3,"gitaly_duration_s":0.226461,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.030973,"redis_read_bytes":460,"redis_write_bytes":29296,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001125,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00121,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.011338,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005292,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15583,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.00695,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000614,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000755,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002564,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.017,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.021,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.233998,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.445043155,"duration_s":0.556701,"completed_at":"2026-09-14T09:31:35.448Z","load_balancing_strategy":"primary","db_duration_s":0.074056,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1847gitlab # [ 242.212515] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.450Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","30a78e656f1740d433631748372c3a791991c03c a4867eecf06d5de25f006944960c955ecd839744 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ff6b844e2a6e2a8e9a5d3967","created_at":"2026-09-14T09:31:32.124Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-b773be3e283043f6","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:31:32.125Z","job_size_bytes":185,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ff6b844e2a6e2a8e9a5d3967: start","job_status":"start","queue_duration_s":3.324743,"scheduling_latency_s":3.32477}1848gitlab # [ 242.213227] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.461Z","message":"changes: \"30a78e656f1740d433631748372c3a791991c03c a4867eecf06d5de25f006944960c955ecd839744 refs/heads/master\\n\""}1849gitlab # [ 242.213893] gitaly[905]: time="2026-09-14T09:31:35.485Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:45.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:35.475" grpc.time_ms=10.701 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 242.263844] gitaly[905]: time="2026-09-14T09:31:35.546Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T09:31:45.069" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:35.539" grpc.time_ms=6.881 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab # [ 242.281663] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.564Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ccac7c59e1d216ca28250122","created_at":"2026-09-14T09:31:35.562Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"7af877c619115497a190d20b","job_size_bytes":46,"pid":2506,"job_status":"deduplicated","message":"ProjectCacheWorker JID-ccac7c59e1d216ca28250122: deduplicated: until executing","deduplication.type":"until executing"}1852gitlab # [ 242.303427] gitaly[905]: time="2026-09-14T09:31:35.586Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-14T09:31:45.105" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:35.576" grpc.time_ms=9.884 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853gitlab # [ 242.325341] gitaly[905]: time="2026-09-14T09:31:35.608Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=656 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-14T09:32:05.598" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:35.598" grpc.time_ms=9.317 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1854gitlab # [ 242.345667] gitaly[905]: time="2026-09-14T09:31:35.628Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-14T09:31:45.150" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-14T09:31:35.621" grpc.time_ms=7.442 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 242.365425] gitaly[905]: time="2026-09-14T09:31:35.648Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:45.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:35.640" grpc.time_ms=7.435 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 242.486196] gitaly[905]: time="2026-09-14T09:31:35.768Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-14T09:31:45.136" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:35.766" grpc.time_ms=1.393 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 242.516277] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.797Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","30a78e656f1740d433631748372c3a791991c03c a4867eecf06d5de25f006944960c955ecd839744 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ff6b844e2a6e2a8e9a5d3967","created_at":"2026-09-14T09:31:32.124Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-b773be3e283043f6","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T09:31:32.125Z","job_size_bytes":185,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ff6b844e2a6e2a8e9a5d3967: done: 0.347377 sec","job_status":"done","queue_duration_s":3.324743,"scheduling_latency_s":3.32477,"gitaly_calls":7,"gitaly_duration_s":0.154402,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.024437,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.003174,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001241,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002916,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007688,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009418,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.029,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.029,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.160015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000445029,"exclusive_lock_hold_duration_s":0.067904161,"duration_s":0.347377,"completed_at":"2026-09-14T09:31:35.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026954,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1858gitlab # [ 242.575272] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.857Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7af877c619115497a190d20b","created_at":"2026-09-14T09:31:33.720Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:33.748Z","job_size_bytes":46,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7af877c619115497a190d20b: start","job_status":"start","queue_duration_s":2.109138,"scheduling_latency_s":2.109185}1859gitlab # [ 242.631705] gitaly[905]: time="2026-09-14T09:31:35.914Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-14T15:31:35.908" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:35.909" grpc.time_ms=4.907 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab # [ 242.657869] gitaly[905]: time="2026-09-14T09:31:35.940Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-14T09:32:05.029" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:35.930" grpc.time_ms=9.833 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861gitlab # [ 242.706165] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.987Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7af877c619115497a190d20b","created_at":"2026-09-14T09:31:33.720Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:33.748Z","job_size_bytes":46,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7af877c619115497a190d20b: done: 0.130466 sec","job_status":"done","queue_duration_s":2.109138,"scheduling_latency_s":2.109185,"gitaly_calls":2,"gitaly_duration_s":0.034465,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010215,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001885,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000903,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002063,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004383,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000520457,"duration_s":0.130466,"completed_at":"2026-09-14T09:31:35.987Z","load_balancing_strategy":"primary","db_duration_s":0.015337,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1862gitlab # [ 242.766895] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:35.990Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bafd41f0e241bd50365899e5","created_at":"2026-09-14T09:31:33.767Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:33.774Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bafd41f0e241bd50365899e5: start","job_status":"start","queue_duration_s":2.216325,"scheduling_latency_s":2.216358}1863gitlab # [ 242.767852] gitaly[905]: time="2026-09-14T09:31:36.044Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 diskcache=70241d67-86e2-4564-9ae0-c86728c23bbb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T15:31:36.016" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.017" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864gitlab # [ 242.768841] gitaly[905]: time="2026-09-14T09:31:36.044Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1278 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T15:31:36.016" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.017" grpc.time_ms=26.954 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 242.840960] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:36.123Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bafd41f0e241bd50365899e5","created_at":"2026-09-14T09:31:33.767Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:33.774Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bafd41f0e241bd50365899e5: done: 0.132992 sec","job_status":"done","queue_duration_s":2.216325,"scheduling_latency_s":2.216358,"gitaly_calls":1,"gitaly_duration_s":0.074557,"redis_calls":15,"redis_duration_s":0.004251,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000552,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000793,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.109139848,"duration_s":0.132992,"completed_at":"2026-09-14T09:31:36.123Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1866gitlab # [ 242.843493] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:36.124Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"3a9292c6f5f4840e36359ee1","created_at":"2026-09-14T09:31:33.968Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:33.970Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3a9292c6f5f4840e36359ee1: start","job_status":"start","queue_duration_s":2.154002,"scheduling_latency_s":2.154018}1867gitlab # [ 242.929308] gitaly[905]: time="2026-09-14T09:31:36.212Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-14T09:32:06.199" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.199" grpc.time_ms=12.13 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 242.953906] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:36.235Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"3a9292c6f5f4840e36359ee1","created_at":"2026-09-14T09:31:33.968Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:33.970Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3a9292c6f5f4840e36359ee1: done: 0.110723 sec","job_status":"done","queue_duration_s":2.154002,"scheduling_latency_s":2.154018,"gitaly_calls":1,"gitaly_duration_s":0.015565,"redis_calls":16,"redis_duration_s":0.060401,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000339,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.012033,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.046729,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.110723,"completed_at":"2026-09-14T09:31:36.235Z","load_balancing_strategy":"primary","db_duration_s":0.003822,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1869gitlab # [ 242.996240] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:36.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"00c36af2d2f62e4f70d8f753","created_at":"2026-09-14T09:31:34.514Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:894d7a0aef2aff0200c8da39b7d8fe4d4f7b51b9bca05a62472db85e7309c55e","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:34.522Z","job_size_bytes":467,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-00c36af2d2f62e4f70d8f753: start","job_status":"start","queue_duration_s":1.755883,"scheduling_latency_s":1.75593}1870gitlab # [ 243.048564] gitaly[905]: time="2026-09-14T09:31:36.331Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:06.329" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.330" grpc.time_ms=0.913 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1871gitlab # [ 243.074501] gitaly[905]: time="2026-09-14T09:31:36.357Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T09:31:46.214" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:36.345" grpc.time_ms=12.011 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1872gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1873gitlab # [ 243.167196] gitaly[905]: time="2026-09-14T09:31:36.449Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:06.447" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.447" grpc.time_ms=1.095 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.06 seconds)1875gitlab # [ 243.194686] gitaly[905]: time="2026-09-14T09:31:36.453Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=25 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-14T09:32:06.451" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.451" grpc.time_ms=1.927 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 243.217887] nixbot[2634]: INFO: ::ffff:127.0.0.1:56158 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1877gitlab # [ 243.237784] gitaly[905]: time="2026-09-14T09:31:36.520Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:46.039" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:36.509" grpc.time_ms=10.933 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 243.607500] gitaly[905]: time="2026-09-14T09:31:36.889Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-14T09:32:06.872" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.872" grpc.time_ms=17.026 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 243.624591] gitaly[905]: time="2026-09-14T09:31:36.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM2EW4CYQGKCPGTDMP2789 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-14T09:32:06.891" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:36.892" grpc.time_ms=14.934 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 243.659360] sidekiq-cluster[2506]: {"severity":"WARN","time":"2026-09-14T09:31:36.941Z","class":"CreatePipelineWorker","correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"30a78e656f1740d433631748372c3a791991c03c","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"30a78e656f1740d433631748372c3a791991c03c","ref":"refs/heads/master"},"retry":0}1881gitlab # [ 243.666806] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:36.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"00c36af2d2f62e4f70d8f753","created_at":"2026-09-14T09:31:34.514Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:894d7a0aef2aff0200c8da39b7d8fe4d4f7b51b9bca05a62472db85e7309c55e","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:34.522Z","job_size_bytes":467,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-00c36af2d2f62e4f70d8f753: done: 0.670793 sec","job_status":"done","queue_duration_s":1.755883,"scheduling_latency_s":1.75593,"gitaly_calls":7,"gitaly_duration_s":0.086023,"redis_calls":51,"redis_duration_s":0.021615,"redis_read_bytes":169,"redis_write_bytes":4166,"redis_cache_calls":4,"redis_cache_duration_s":0.001105,"redis_cache_read_bytes":10,"redis_cache_write_bytes":341,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004111,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.013691,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.487888,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000408432,"duration_s":0.670793,"completed_at":"2026-09-14T09:31:36.949Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1882gitlab # [ 243.671157] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:36.951Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6e1841fc-5147-4811-9a10-da2ea16acf23\"}"],"class":"WebHookWorker","jid":"2645da29e09656a1a008d13b","created_at":"2026-09-14T09:31:34.661Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ecd27c358c4af3f78b9ac89eb38c45fcc841575e06be2027534efd1a461e0a95","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:34.668Z","job_size_bytes":1751,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2645da29e09656a1a008d13b: start","job_status":"start","queue_duration_s":2.283401,"scheduling_latency_s":2.283428}1883gitlab # [ 243.821720] nixbot[2634]: INFO: ::ffff:127.0.0.1:56164 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1884gitlab # [ 243.836734] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.119Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6e1841fc-5147-4811-9a10-da2ea16acf23\"}"],"class":"WebHookWorker","jid":"2645da29e09656a1a008d13b","created_at":"2026-09-14T09:31:34.661Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-4bdb1555012f4ae1","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ecd27c358c4af3f78b9ac89eb38c45fcc841575e06be2027534efd1a461e0a95","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:34.668Z","job_size_bytes":1751,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-2645da29e09656a1a008d13b: done: 0.1677 sec","job_status":"done","queue_duration_s":2.283401,"scheduling_latency_s":2.283428,"redis_calls":19,"redis_duration_s":0.004527,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000947,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000351,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002149,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.057468756999981,"cpu_s":0.061026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000373232,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.1677,"completed_at":"2026-09-14T09:31:37.119Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1885gitlab # [ 243.860163] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.120Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"bab28011e27c54f673c03d17","created_at":"2026-09-14T09:31:35.028Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.031Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-bab28011e27c54f673c03d17: start","job_status":"start","queue_duration_s":2.089243,"scheduling_latency_s":2.089259}1886gitlab # [ 243.898144] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"bab28011e27c54f673c03d17","created_at":"2026-09-14T09:31:35.028Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.031Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-bab28011e27c54f673c03d17: done: 0.059886 sec","job_status":"done","queue_duration_s":2.089243,"scheduling_latency_s":2.089259,"redis_calls":10,"redis_duration_s":0.0024619999999999998,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001488,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000300038,"duration_s":0.059886,"completed_at":"2026-09-14T09:31:37.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017967,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1887gitlab # [ 243.899559] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.182Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f3f4d263e7b8ca4d76c8e600","created_at":"2026-09-14T09:31:35.237Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.243Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f3f4d263e7b8ca4d76c8e600: start","job_status":"start","queue_duration_s":1.93864,"scheduling_latency_s":1.938656}1888gitlab # [ 243.941748] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.205Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f3f4d263e7b8ca4d76c8e600","created_at":"2026-09-14T09:31:35.237Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.243Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f3f4d263e7b8ca4d76c8e600: done: 0.023527 sec","job_status":"done","queue_duration_s":1.93864,"scheduling_latency_s":1.938656,"redis_calls":10,"redis_duration_s":0.0019130000000000002,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000366,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000262324,"duration_s":0.023527,"completed_at":"2026-09-14T09:31:37.205Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005881,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab # [ 243.984126] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.206Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"80590789c855ca0f022135b0","created_at":"2026-09-14T09:31:35.259Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.264Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-80590789c855ca0f022135b0: start","job_status":"start","queue_duration_s":1.941955,"scheduling_latency_s":1.941968}1890gitlab # [ 243.985282] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.230Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"80590789c855ca0f022135b0","created_at":"2026-09-14T09:31:35.259Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.264Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-80590789c855ca0f022135b0: done: 0.023479 sec","job_status":"done","queue_duration_s":1.941955,"scheduling_latency_s":1.941968,"redis_calls":10,"redis_duration_s":0.002227,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000266235,"duration_s":0.023479,"completed_at":"2026-09-14T09:31:37.230Z","load_balancing_strategy":"primary","db_duration_s":0.005155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1891gitlab # [ 244.033724] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.231Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5afe0420e3c79bd98155a839","created_at":"2026-09-14T09:31:35.280Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.289Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5afe0420e3c79bd98155a839: start","job_status":"start","queue_duration_s":1.941891,"scheduling_latency_s":1.941906}1892gitlab # [ 244.034834] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.265Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"5afe0420e3c79bd98155a839","created_at":"2026-09-14T09:31:35.280Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.289Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-5afe0420e3c79bd98155a839: done: 0.03462 sec","job_status":"done","queue_duration_s":1.941891,"scheduling_latency_s":1.941906,"redis_calls":13,"redis_duration_s":0.002431,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000549,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000293892,"duration_s":0.03462,"completed_at":"2026-09-14T09:31:37.265Z","load_balancing_strategy":"primary","db_duration_s":0.005154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1893gitlab # [ 244.066692] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c4be57543a58a39a05327b72","created_at":"2026-09-14T09:31:35.290Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.297Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c4be57543a58a39a05327b72: start","job_status":"start","queue_duration_s":1.969759,"scheduling_latency_s":1.969773}1894gitlab # [ 244.106343] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c4be57543a58a39a05327b72","created_at":"2026-09-14T09:31:35.290Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.297Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c4be57543a58a39a05327b72: done: 0.022719 sec","job_status":"done","queue_duration_s":1.969759,"scheduling_latency_s":1.969773,"redis_calls":10,"redis_duration_s":0.001919,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000372,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000298641,"duration_s":0.022719,"completed_at":"2026-09-14T09:31:37.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1895gitlab # [ 244.122274] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4bd777c4334c0f3878dfed17","created_at":"2026-09-14T09:31:35.298Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.305Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4bd777c4334c0f3878dfed17: start","job_status":"start","queue_duration_s":1.985539,"scheduling_latency_s":1.985594}1896gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1897gitlab # [ 244.137717] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.312Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4bd777c4334c0f3878dfed17","created_at":"2026-09-14T09:31:35.298Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.305Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4bd777c4334c0f3878dfed17: done: 0.021238 sec","job_status":"done","queue_duration_s":1.985539,"scheduling_latency_s":1.985594,"redis_calls":11,"redis_duration_s":0.002399,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001522,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310375,"duration_s":0.021238,"completed_at":"2026-09-14T09:31:37.312Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1898gitlab # [ 244.178251] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.313Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7f9a9288ca6bf6a9ddc3bd49","created_at":"2026-09-14T09:31:35.309Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.314Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7f9a9288ca6bf6a9ddc3bd49: start","job_status":"start","queue_duration_s":1.998645,"scheduling_latency_s":1.998659}1899gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.05 seconds)1900gitlab # [ 244.225738] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.366Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7f9a9288ca6bf6a9ddc3bd49","created_at":"2026-09-14T09:31:35.309Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.314Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7f9a9288ca6bf6a9ddc3bd49: done: 0.053109 sec","job_status":"done","queue_duration_s":1.998645,"scheduling_latency_s":1.998659,"redis_calls":10,"redis_duration_s":0.0018410000000000002,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002288,"duration_s":0.053109,"completed_at":"2026-09-14T09:31:37.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1901gitlab # [ 244.271227] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.367Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"306e37548836f253981af962","created_at":"2026-09-14T09:31:35.316Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.321Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-306e37548836f253981af962: start","job_status":"start","queue_duration_s":2.046503,"scheduling_latency_s":2.046518}1902gitlab # [ 244.271983] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.391Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"306e37548836f253981af962","created_at":"2026-09-14T09:31:35.316Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.321Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-306e37548836f253981af962: done: 0.024171 sec","job_status":"done","queue_duration_s":2.046503,"scheduling_latency_s":2.046518,"redis_calls":10,"redis_duration_s":0.0013679999999999999,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243327,"duration_s":0.024171,"completed_at":"2026-09-14T09:31:37.391Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003148,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1903gitlab # [ 244.313520] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.392Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a94d933f503e7ce3eb2dad12","created_at":"2026-09-14T09:31:35.329Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.332Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a94d933f503e7ce3eb2dad12: start","job_status":"start","queue_duration_s":2.059984,"scheduling_latency_s":2.059997}1904gitlab # [ 244.314137] nixbot[2634]: INFO: ::ffff:127.0.0.1:56176 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1905gitlab # [ 244.314601] gitaly[905]: time="2026-09-14T09:31:37.588Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 diskcache=a19a0a6f-152e-48dc-9c79-e0ad50202d52 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T15:31:37.578" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:37.578" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 244.315130] gitaly[905]: time="2026-09-14T09:31:37.588Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T15:31:37.578" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:37.578" grpc.time_ms=9.796 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 244.326131] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.416Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a94d933f503e7ce3eb2dad12","created_at":"2026-09-14T09:31:35.329Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.332Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a94d933f503e7ce3eb2dad12: done: 0.023336 sec","job_status":"done","queue_duration_s":2.059984,"scheduling_latency_s":2.059997,"redis_calls":10,"redis_duration_s":0.001403,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000223772,"duration_s":0.023336,"completed_at":"2026-09-14T09:31:37.416Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1908gitlab # [ 244.372082] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.417Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"da23c98451caa210efde23f4","created_at":"2026-09-14T09:31:35.333Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.336Z","job_size_bytes":6,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-da23c98451caa210efde23f4: start","job_status":"start","queue_duration_s":2.080736,"scheduling_latency_s":2.080749}1909gitlab # [ 244.379766] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.431Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"da23c98451caa210efde23f4","created_at":"2026-09-14T09:31:35.333Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.336Z","job_size_bytes":6,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-da23c98451caa210efde23f4: done: 0.013988 sec","job_status":"done","queue_duration_s":2.080736,"scheduling_latency_s":2.080749,"redis_calls":10,"redis_duration_s":0.001371,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208407,"duration_s":0.013988,"completed_at":"2026-09-14T09:31:37.431Z","load_balancing_strategy":"primary","db_duration_s":0.001336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1910gitlab # [ 244.436508] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.432Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"da3ea18dfaff7858019bf642","created_at":"2026-09-14T09:31:35.336Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.339Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-da3ea18dfaff7858019bf642: start","job_status":"start","queue_duration_s":2.092552,"scheduling_latency_s":2.092565}1911gitlab # [ 244.437314] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.452Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"da3ea18dfaff7858019bf642","created_at":"2026-09-14T09:31:35.336Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.339Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-da3ea18dfaff7858019bf642: done: 0.020727 sec","job_status":"done","queue_duration_s":2.092552,"scheduling_latency_s":2.092565,"redis_calls":10,"redis_duration_s":0.001364,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000248076,"duration_s":0.020727,"completed_at":"2026-09-14T09:31:37.452Z","load_balancing_strategy":"primary","db_duration_s":0.00361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1912gitlab # [ 244.473332] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.453Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0a5783b1f708f0121a65fd4c","created_at":"2026-09-14T09:31:35.344Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.347Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0a5783b1f708f0121a65fd4c: start","job_status":"start","queue_duration_s":2.106145,"scheduling_latency_s":2.10616}1913gitlab # [ 244.518691] gitaly[905]: time="2026-09-14T09:31:37.680Z" level=info msg="finished unary call with code OK" catfile.duration_ms=57 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=57 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:07.622" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:37.622" grpc.time_ms=58.088 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 244.519604] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.547Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0a5783b1f708f0121a65fd4c","created_at":"2026-09-14T09:31:35.344Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.347Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0a5783b1f708f0121a65fd4c: done: 0.093685 sec","job_status":"done","queue_duration_s":2.106145,"scheduling_latency_s":2.10616,"redis_calls":14,"redis_duration_s":0.001991,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000245283,"duration_s":0.093685,"completed_at":"2026-09-14T09:31:37.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1915gitlab # [ 244.538121] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.548Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"202780ed9afd9bdec4faf369","created_at":"2026-09-14T09:31:35.436Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.438Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-202780ed9afd9bdec4faf369: start","job_status":"start","queue_duration_s":2.110026,"scheduling_latency_s":2.110045}1916gitlab # [ 244.579096] gitaly[905]: time="2026-09-14T09:31:37.689Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T09:31:47.213" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:37.684" grpc.time_ms=5.492 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1917gitlab # [ 244.579775] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.566Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"202780ed9afd9bdec4faf369","created_at":"2026-09-14T09:31:35.436Z","trace_propagation_headers":{"sentry-trace":"85a93ec5a54f4094b7e8cf5fe5263701-0ae622eaa7474563","baggage":"sentry-trace_id=85a93ec5a54f4094b7e8cf5fe5263701,sentry-release=Unknown"},"correlation_id":"01M2FM2KQRVPT0Z697A4EQRY7V","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.438Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-202780ed9afd9bdec4faf369: done: 0.017661 sec","job_status":"done","queue_duration_s":2.110026,"scheduling_latency_s":2.110045,"redis_calls":8,"redis_duration_s":0.00118,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017661,"completed_at":"2026-09-14T09:31:37.566Z","load_balancing_strategy":"primary","db_duration_s":0.003131,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1918gitlab # [ 244.600491] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.567Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bdcd7fab9f8bb8952648ec3c","created_at":"2026-09-14T09:31:35.570Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"bafd41f0e241bd50365899e5","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.572Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bdcd7fab9f8bb8952648ec3c: start","job_status":"start","queue_duration_s":1.995279,"scheduling_latency_s":1.995295}1919gitlab # [ 244.638662] gitaly[905]: time="2026-09-14T09:31:37.713Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:07.712" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:37.712" grpc.time_ms=0.646 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 244.639559] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.604Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bdcd7fab9f8bb8952648ec3c","created_at":"2026-09-14T09:31:35.570Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"bafd41f0e241bd50365899e5","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.572Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bdcd7fab9f8bb8952648ec3c: done: 0.036753 sec","job_status":"done","queue_duration_s":1.995279,"scheduling_latency_s":1.995295,"gitaly_calls":1,"gitaly_duration_s":0.011885,"redis_calls":12,"redis_duration_s":0.0027010000000000003,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000398,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000844,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.023521425,"duration_s":0.036753,"completed_at":"2026-09-14T09:31:37.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1921gitlab # [ 244.680034] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:37.605Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5ab48940d900c19db79b9d05","created_at":"2026-09-14T09:31:35.724Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:12b1d5117dcfa7953b27dcdae8b7af8d6044722efa3f032b75afcfd51fdc984a","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.728Z","job_size_bytes":467,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5ab48940d900c19db79b9d05: start","job_status":"start","queue_duration_s":1.877358,"scheduling_latency_s":1.877376}1922gitlab # [ 244.680976] gitaly[905]: time="2026-09-14T09:31:37.718Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=99 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-14T09:32:07.716" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:37.716" grpc.time_ms=1.82 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 244.726464] gitaly[905]: time="2026-09-14T09:31:37.769Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:47.262" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:37.732" grpc.time_ms=36.082 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 244.777653] gitaly[905]: time="2026-09-14T09:31:38.060Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3Z9HPZCAT08MRDP5AWZR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.052" grpc.time_ms=7.854 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1925gitlab # [ 244.790658] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 9231926gitlab # [ 244.812425] gitaly[905]: time="2026-09-14T09:31:38.095Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-14T09:32:08.084" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.084" grpc.time_ms=10.773 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 244.838079] gitaly[905]: time="2026-09-14T09:31:38.120Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM406F0F0S9HWSZDYK27SZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.114" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1928gitlab # [ 244.838508] gitaly[905]: time="2026-09-14T09:31:38.121Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM406F0F0S9HWSZDYK27SZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.114" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1929gitlab # [ 244.839461] gitaly[905]: time="2026-09-14T09:31:38.121Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM406F0F0S9HWSZDYK27SZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.114" grpc.time_ms=6.957 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1930gitlab # [ 244.840198] gitlab-workhorse[832]: time="2026-09-14T09:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671931gitlab # [ 244.840662] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 581932gitlab # [ 244.845831] gitaly[905]: time="2026-09-14T09:31:38.128Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM3SYJVVJY2QF8TKFFM750 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-14T09:32:08.109" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.109" grpc.time_ms=18.873 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933gitlab # [ 244.857769] sidekiq-cluster[2506]: {"severity":"WARN","time":"2026-09-14T09:31:38.140Z","class":"CreatePipelineWorker","correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"30a78e656f1740d433631748372c3a791991c03c","after":"a4867eecf06d5de25f006944960c955ecd839744","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a4867eecf06d5de25f006944960c955ecd839744","ref":"refs/heads/master"},"retry":0}1934gitlab # [ 244.862098] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.144Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5ab48940d900c19db79b9d05","created_at":"2026-09-14T09:31:35.724Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:12b1d5117dcfa7953b27dcdae8b7af8d6044722efa3f032b75afcfd51fdc984a","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.728Z","job_size_bytes":467,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5ab48940d900c19db79b9d05: done: 0.538974 sec","job_status":"done","queue_duration_s":1.877358,"scheduling_latency_s":1.877376,"gitaly_calls":7,"gitaly_duration_s":0.158725,"redis_calls":33,"redis_duration_s":0.004808,"redis_read_bytes":184,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000498,"redis_cache_read_bytes":43,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001987,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.360809,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.538974,"completed_at":"2026-09-14T09:31:38.144Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014608,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1935gitlab # [ 244.918808] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.150Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"10af60de-8310-452b-bba4-c7af0978475a\"}"],"class":"WebHookWorker","jid":"129e047a9644c117826cce1c","created_at":"2026-09-14T09:31:35.748Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d337bc72f9f7e2466539e70c269b470b795c1cbe554830ce6cc90ff00d66f6b8","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.751Z","job_size_bytes":1749,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-129e047a9644c117826cce1c: start","job_status":"start","queue_duration_s":2.398962,"scheduling_latency_s":2.398979}1936gitlab # [ 244.919688] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.183Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"10af60de-8310-452b-bba4-c7af0978475a\"}"],"class":"WebHookWorker","jid":"129e047a9644c117826cce1c","created_at":"2026-09-14T09:31:35.748Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-4b0ab50ee5164588","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d337bc72f9f7e2466539e70c269b470b795c1cbe554830ce6cc90ff00d66f6b8","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.751Z","job_size_bytes":1749,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-129e047a9644c117826cce1c: done: 0.032821 sec","job_status":"done","queue_duration_s":2.398962,"scheduling_latency_s":2.398979,"redis_calls":14,"redis_duration_s":0.004221,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000388,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002869,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007007035999976097,"cpu_s":0.014657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.032821,"completed_at":"2026-09-14T09:31:38.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1937gitlab # [ 244.962550] nixbot[2634]: INFO: ::ffff:127.0.0.1:56180 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1938gitlab # [ 244.963235] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.184Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"114ad14609c5b87f44876721","created_at":"2026-09-14T09:31:35.966Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-9c69f16530dc4d3f","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.972Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-114ad14609c5b87f44876721: start","job_status":"start","queue_duration_s":2.211972,"scheduling_latency_s":2.211986}1939gitlab # [ 245.007534] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.220Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"114ad14609c5b87f44876721","created_at":"2026-09-14T09:31:35.966Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-9c69f16530dc4d3f","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:35.972Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-114ad14609c5b87f44876721: done: 0.035988 sec","job_status":"done","queue_duration_s":2.211972,"scheduling_latency_s":2.211986,"redis_calls":19,"redis_duration_s":0.006106,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001188,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00047,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003749,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000699,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005990147,"duration_s":0.035988,"completed_at":"2026-09-14T09:31:38.220Z","load_balancing_strategy":"primary","db_duration_s":0.008931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1940gitlab # [ 245.054675] gitaly[905]: time="2026-09-14T09:31:38.281Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40B1F0FZ48V0NG2VET1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.267" pid=905 remote_ip="2001:db8:1::1" request_sha=1e7517af177f68958fe7f26336c4e6a366855727 response_bytes=337 span.kind=server system=grpc1941gitlab # [ 245.055513] nixbot[2634]: {"ts": "2026-09-14T09:31:38.327Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1942gitlab # [ 245.056076] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.221Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2645da29e09656a1a008d13b"],"class":"WebHooks::LogExecutionWorker","jid":"bf9911a4b66da67f0acdfa8c","created_at":"2026-09-14T09:31:37.108Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-3b3d7b48d0764228","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:432a1e4dd8e29640df686a1234a6c79136758efe81f5532c155f6e5531131148","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:37.113Z","job_size_bytes":2592,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-bf9911a4b66da67f0acdfa8c: start","job_status":"start","queue_duration_s":1.108188,"scheduling_latency_s":1.108203}1943gitlab # [ 245.056769] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.272Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","2645da29e09656a1a008d13b"],"class":"WebHooks::LogExecutionWorker","jid":"bf9911a4b66da67f0acdfa8c","created_at":"2026-09-14T09:31:37.108Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-3b3d7b48d0764228","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:432a1e4dd8e29640df686a1234a6c79136758efe81f5532c155f6e5531131148","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:37.113Z","job_size_bytes":2592,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-bf9911a4b66da67f0acdfa8c: done: 0.05064 sec","job_status":"done","queue_duration_s":1.108188,"scheduling_latency_s":1.108203,"redis_calls":10,"redis_duration_s":0.005478,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001745,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000616,"duration_s":0.05064,"completed_at":"2026-09-14T09:31:38.272Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 245.091326] gitlab-workhorse[832]: time="2026-09-14T09:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371945gitlab # [ 245.092026] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 721946gitlab # [ 245.095704] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 451947gitlab # [ 245.096573] gitaly[905]: time="2026-09-14T09:31:38.281Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1189 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40B1F0FZ48V0NG2VET1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.267" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1948gitlab # [ 245.138546] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.274Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","129e047a9644c117826cce1c"],"class":"WebHooks::LogExecutionWorker","jid":"4328ab0328679caf39df80dc","created_at":"2026-09-14T09:31:38.178Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-f4b9b3503da94e32","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c195fe0e72313516406f2d626f695dc9b082ae4cef233d94e40aedc6746b55f3","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.179Z","job_size_bytes":2592,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4328ab0328679caf39df80dc: start","job_status":"start","queue_duration_s":0.094359,"scheduling_latency_s":0.094378}1949gitlab # [ 245.138770] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.302Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","129e047a9644c117826cce1c"],"class":"WebHooks::LogExecutionWorker","jid":"4328ab0328679caf39df80dc","created_at":"2026-09-14T09:31:38.178Z","trace_propagation_headers":{"sentry-trace":"943a7c4a20574785af42968bdf88599c-f4b9b3503da94e32","baggage":"sentry-trace_id=943a7c4a20574785af42968bdf88599c,sentry-release=Unknown"},"correlation_id":"01M2FM3SYJVVJY2QF8TKFFM750","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c195fe0e72313516406f2d626f695dc9b082ae4cef233d94e40aedc6746b55f3","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.179Z","job_size_bytes":2592,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4328ab0328679caf39df80dc: done: 0.028477 sec","job_status":"done","queue_duration_s":0.094359,"scheduling_latency_s":0.094378,"redis_calls":7,"redis_duration_s":0.001716,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028477,"completed_at":"2026-09-14T09:31:38.302Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1950gitlab # [ 245.181958] gitlab-workhorse[832]: time="2026-09-14T09:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671951gitlab # [ 245.182663] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 471952gitlab # [ 245.183222] gitaly[905]: time="2026-09-14T09:31:38.281Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1189 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40B1F0FZ48V0NG2VET1Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.267" grpc.time_ms=13.829 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1953gitlab # [ 245.183925] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"28c87159d6d9a1ef723090cc","created_at":"2026-09-14T09:31:38.208Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-343dea4283c44e5f","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.213Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-28c87159d6d9a1ef723090cc: start","job_status":"start","queue_duration_s":0.091798,"scheduling_latency_s":0.09185}1954gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'1955gitlab # [ 245.225278] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.405Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"28c87159d6d9a1ef723090cc","created_at":"2026-09-14T09:31:38.208Z","trace_propagation_headers":{"sentry-trace":"2e1250b12659496a89b4b272df177e98-343dea4283c44e5f","baggage":"sentry-trace_id=2e1250b12659496a89b4b272df177e98,sentry-release=Unknown"},"correlation_id":"01M2FM2EW4CYQGKCPGTDMP2789","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.213Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-28c87159d6d9a1ef723090cc: done: 0.100436 sec","job_status":"done","queue_duration_s":0.091798,"scheduling_latency_s":0.09185,"redis_calls":11,"redis_duration_s":0.00494,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000655,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003382,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025981,"duration_s":0.100436,"completed_at":"2026-09-14T09:31:38.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 245.231924] gitaly[905]: time="2026-09-14T09:31:38.369Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40EMYVDN9NEVENP583RJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.364" grpc.time_ms=5.447 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1957gitlab # [ 245.241359] gitaly[905]: time="2026-09-14T09:31:38.418Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40G3FEAQ9A82940H1B2D grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.411" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1958gitlab # [ 245.250639] nixbot[2634]: {"ts": "2026-09-14T09:31:38.501Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1959gitlab # [ 245.253036] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.490Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4bdbc2557f9cd887bf99342f","created_at":"2026-09-14T09:31:38.486Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-3f2f33e9c76b41fc","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.489Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4bdbc2557f9cd887bf99342f: start","job_status":"start","queue_duration_s":0.001395,"scheduling_latency_s":0.00143}1960gitlab # [ 245.295289] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1601961gitlab # [ 245.295917] gitaly[905]: time="2026-09-14T09:31:38.418Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40G3FEAQ9A82940H1B2D grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.411" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1962gitlab # [ 245.296575] gitaly[905]: time="2026-09-14T09:31:38.419Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40G3FEAQ9A82940H1B2D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.411" grpc.time_ms=7.116 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1963gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.10 seconds)1964gitlab # [ 245.297428] gitaly[905]: time="2026-09-14T09:31:38.426Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:08.423" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.423" grpc.time_ms=2.566 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1965gitlab # [ 245.304979] gitaly[905]: time="2026-09-14T09:31:38.464Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:48.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.458" grpc.time_ms=5.317 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 245.353764] gitaly[905]: time="2026-09-14T09:31:38.578Z" level=info msg="finished unary call with code OK" catfile.duration_ms=20 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=20 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40MHEN19EWAPTYAKD4JR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:08.556" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.556" grpc.time_ms=21.13 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 245.354622] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1791968gitlab # [ 245.355446] nixbot[2634]: INFO: ::ffff:127.0.0.1:56188 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK1969gitlab # [ 245.355997] gitaly[905]: time="2026-09-14T09:31:38.629Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=13 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40JK8QJWSJEYJZ3JSH7R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.609" grpc.time_ms=20.114 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1970gitlab # [ 245.356672] gitaly[905]: time="2026-09-14T09:31:38.632Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40MHEN19EWAPTYAKD4JR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:48.064" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.624" grpc.time_ms=8.197 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971gitlab # [ 245.388987] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1571972gitlab # [ 245.390953] nixbot[2634]: {"ts": "2026-09-14T09:31:38.673Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1973gitlab # [ 245.410621] gitaly[905]: time="2026-09-14T09:31:38.693Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:38.692" grpc.time_ms=0.781 pid=905 span.kind=server system=grpc1974gitlab # [ 245.420597] gitaly[905]: time="2026-09-14T09:31:38.703Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40R8MDDB52718TQKAD8N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.684" pid=905 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1975gitlab # [ 245.420960] gitaly[905]: time="2026-09-14T09:31:38.703Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40R8MDDB52718TQKAD8N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.684" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1976gitlab # [ 245.427577] gitaly[905]: time="2026-09-14T09:31:38.710Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40R8MDDB52718TQKAD8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:38.684" grpc.time_ms=25.408 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc1977gitlab # [ 245.429638] gitlab-workhorse[832]: time="2026-09-14T09:31:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161978gitlab # [ 245.430614] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 781979gitlab # [ 245.460227] gitaly[905]: time="2026-09-14T09:31:38.743Z" level=info msg="finished unary call with code OK" catfile.duration_ms=20 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=20 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40SN59NWQD4PG7FQT16X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:08.721" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.721" grpc.time_ms=21.107 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 245.490366] gitaly[905]: time="2026-09-14T09:31:38.773Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:38.772" grpc.time_ms=0.362 pid=905 span.kind=server system=grpc1981gitlab # [ 245.498354] gitaly[905]: time="2026-09-14T09:31:38.781Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40SN59NWQD4PG7FQT16X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:48.215" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.775" grpc.time_ms=5.693 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982gitlab # [ 245.532808] nixbot[2634]: {"ts": "2026-09-14T09:31:38.815Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c \"HTTP/1.1 201 Created\""}1983gitlab # [ 245.536952] gitaly[905]: time="2026-09-14T09:31:38.819Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:38.816" grpc.time_ms=3.323 pid=905 span.kind=server system=grpc1984gitlab # [ 245.537535] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/30a78e656f1740d433631748372c3a791991c03c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1421985gitlab # [ 245.556926] gitaly[905]: time="2026-09-14T09:31:38.839Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 diskcache=fd685d21-f1d6-404d-a986-2a5cda392724 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:08.597" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.598" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 245.557620] gitaly[905]: time="2026-09-14T09:31:38.840Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=130 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7123 command.oublock=0 command.real_time_ms=234 command.spawn_token_fork_ms=0 command.system_time_ms=101 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:08.597" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.598" grpc.time_ms=241.893 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 245.599727] gitaly[905]: time="2026-09-14T09:31:38.881Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:08.879" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.879" grpc.time_ms=1.751 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab # [ 245.602581] gitaly[905]: time="2026-09-14T09:31:38.885Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:08.380" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.880" grpc.time_ms=4.577 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989gitlab # [ 245.619737] gitaly[905]: time="2026-09-14T09:31:38.902Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-14T09:32:08.393" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.894" grpc.time_ms=8.378 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 245.620689] gitaly[905]: time="2026-09-14T09:31:38.902Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:08.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:38.902" grpc.time_ms=0.673 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1991gitlab # [ 245.625528] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.907Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3993b833348660df2b5fc055","created_at":"2026-09-14T09:31:38.906Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"4bdbc2557f9cd887bf99342f","job_size_bytes":3,"pid":2506,"job_status":"deduplicated","message":"PipelineProcessWorker JID-3993b833348660df2b5fc055: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1992gitlab # [ 245.626527] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.909Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4bdbc2557f9cd887bf99342f","created_at":"2026-09-14T09:31:38.486Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-3f2f33e9c76b41fc","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.489Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2506,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4bdbc2557f9cd887bf99342f: rescheduled"}1993gitlab # [ 245.634951] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.912Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4bdbc2557f9cd887bf99342f","created_at":"2026-09-14T09:31:38.486Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-3f2f33e9c76b41fc","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.489Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4bdbc2557f9cd887bf99342f: done: 0.422519 sec","job_status":"done","queue_duration_s":0.001395,"scheduling_latency_s":0.00143,"gitaly_calls":3,"gitaly_duration_s":0.249824,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.015609999999999999,"redis_read_bytes":226,"redis_write_bytes":18461,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000651,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000638,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004509,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12307,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006931,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000194,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000403,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001633,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.02,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.02,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083301,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.391060317,"duration_s":0.422519,"completed_at":"2026-09-14T09:31:38.912Z","load_balancing_strategy":"primary","db_duration_s":0.072914,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1994gitlab # [ 245.686546] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.914Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c29b3e20526ae0e54d6922c3","created_at":"2026-09-14T09:31:38.585Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.591Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c29b3e20526ae0e54d6922c3: start","job_status":"start","queue_duration_s":0.322468,"scheduling_latency_s":0.322485}1995gitlab # [ 245.699646] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.952Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c29b3e20526ae0e54d6922c3","created_at":"2026-09-14T09:31:38.585Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.591Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c29b3e20526ae0e54d6922c3: done: 0.03806 sec","job_status":"done","queue_duration_s":0.322468,"scheduling_latency_s":0.322485,"redis_calls":7,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03806,"completed_at":"2026-09-14T09:31:38.952Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010283,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1996gitlab # [ 245.766066] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.954Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e14a04254403a2622c0c6e98","created_at":"2026-09-14T09:31:38.844Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.846Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e14a04254403a2622c0c6e98: start","job_status":"start","queue_duration_s":0.107723,"scheduling_latency_s":0.107754}1997gitlab # [ 245.767104] gitaly[905]: time="2026-09-14T09:31:38.919Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:48.012" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.912" grpc.time_ms=6.73 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 245.767694] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.981Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e14a04254403a2622c0c6e98","created_at":"2026-09-14T09:31:38.844Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.846Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e14a04254403a2622c0c6e98: done: 0.027604 sec","job_status":"done","queue_duration_s":0.107723,"scheduling_latency_s":0.107754,"redis_calls":7,"redis_duration_s":0.004074,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027604,"completed_at":"2026-09-14T09:31:38.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 245.809499] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:38.986Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"837668e1d42a90b2ca82118a","created_at":"2026-09-14T09:31:38.852Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.854Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-837668e1d42a90b2ca82118a: start","job_status":"start","queue_duration_s":0.131425,"scheduling_latency_s":0.131445}2000gitlab # [ 245.852758] gitaly[905]: time="2026-09-14T09:31:38.953Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T09:31:48.052" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:38.953" grpc.time_ms=0.217 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001gitlab # [ 245.853593] nixbot[2634]: {"ts": "2026-09-14T09:31:39.103Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 \"HTTP/1.1 201 Created\""}2002gitlab # [ 245.854203] nixbot[2634]: {"ts": "2026-09-14T09:31:39.111Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-a4867eecf06d-4"}2003gitlab # [ 245.854894] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.012Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"837668e1d42a90b2ca82118a","created_at":"2026-09-14T09:31:38.852Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.854Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-837668e1d42a90b2ca82118a: done: 0.026751 sec","job_status":"done","queue_duration_s":0.131425,"scheduling_latency_s":0.131445,"redis_calls":7,"redis_duration_s":0.003951,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026751,"completed_at":"2026-09-14T09:31:39.012Z","load_balancing_strategy":"primary","db_duration_s":0.007014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 245.872932] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.013Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"daaa0ae587971f75633526cb","created_at":"2026-09-14T09:31:38.860Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.861Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-daaa0ae587971f75633526cb: start","job_status":"start","queue_duration_s":0.152231,"scheduling_latency_s":0.152246}2005gitlab # [ 245.914946] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2932006gitlab # [ 245.915725] gitaly[905]: time="2026-09-14T09:31:38.964Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T09:31:48.057" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:38.957" grpc.time_ms=6.511 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2007gitlab # [ 245.917596] gitaly[905]: time="2026-09-14T09:31:38.994Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:38.993" grpc.time_ms=0.57 pid=905 span.kind=server system=grpc2008gitlab # [ 245.918638] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.032Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"daaa0ae587971f75633526cb","created_at":"2026-09-14T09:31:38.860Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.861Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-daaa0ae587971f75633526cb: done: 0.018473 sec","job_status":"done","queue_duration_s":0.152231,"scheduling_latency_s":0.152246,"redis_calls":10,"redis_duration_s":0.001412,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000388,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000321,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018473,"completed_at":"2026-09-14T09:31:39.032Z","load_balancing_strategy":"primary","db_duration_s":0.00321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2009gitlab # [ 245.967784] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.033Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4607f979eecad3a1bfb96eac","created_at":"2026-09-14T09:31:38.862Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.863Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4607f979eecad3a1bfb96eac: start","job_status":"start","queue_duration_s":0.169554,"scheduling_latency_s":0.169569}2010gitlab # [ 245.968683] gitaly[905]: time="2026-09-14T09:31:39.012Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4151 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:39.012" grpc.time_ms=0.231 pid=905 span.kind=server system=grpc2011gitlab # [ 246.008330] gitaly[905]: time="2026-09-14T09:31:39.021Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS diskcache=75f12fd1-4501-49fd-9e0b-0979cccf48fd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T09:31:48.065" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:38.966" pid=905 ref=refs/keep-around/a4867eecf06d5de25f006944960c955ecd839744 remote_ip="2001:db8:1::1" revision=a4867eecf06d5de25f006944960c955ecd839744 span.kind=server system=grpc username=root2012gitlab # [ 246.009132] nixbot[2634]: {"ts": "2026-09-14T09:31:39.233Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a4867eecf06d-4", "argv0": "bwrap"}2013gitlab # [ 246.009605] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"4607f979eecad3a1bfb96eac","created_at":"2026-09-14T09:31:38.862Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.863Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-4607f979eecad3a1bfb96eac: done: 0.014906 sec","job_status":"done","queue_duration_s":0.169554,"scheduling_latency_s":0.169569,"redis_calls":7,"redis_duration_s":0.001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014906,"completed_at":"2026-09-14T09:31:39.047Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 246.053707] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"b0112542470542f8630a28ed","created_at":"2026-09-14T09:31:38.864Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.866Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-b0112542470542f8630a28ed: start","job_status":"start","queue_duration_s":0.183881,"scheduling_latency_s":0.183926}2015gitlab # [ 246.055092] gitaly[905]: time="2026-09-14T09:31:39.021Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=43 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5050 command.oublock=8 command.real_time_ms=46 command.spawn_token_fork_ms=0 command.system_time_ms=30 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T09:31:48.065" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T09:31:38.966" grpc.time_ms=54.875 pid=905 ref=refs/keep-around/a4867eecf06d5de25f006944960c955ecd839744 remote_ip="2001:db8:1::1" revision=a4867eecf06d5de25f006944960c955ecd839744 span.kind=server system=grpc username=root2016gitlab # [ 246.055806] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.069Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"b0112542470542f8630a28ed","created_at":"2026-09-14T09:31:38.864Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.866Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-b0112542470542f8630a28ed: done: 0.019949 sec","job_status":"done","queue_duration_s":0.183881,"scheduling_latency_s":0.183926,"redis_calls":8,"redis_duration_s":0.004507000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004172,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019949,"completed_at":"2026-09-14T09:31:39.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 246.097575] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.070Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b20dd5d2f3c85ca83d1faac1","created_at":"2026-09-14T09:31:38.866Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.867Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b20dd5d2f3c85ca83d1faac1: start","job_status":"start","queue_duration_s":0.202944,"scheduling_latency_s":0.202959}2018gitlab # [ 246.140240] gitaly[905]: time="2026-09-14T09:31:39.024Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:09.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:39.024" grpc.time_ms=0.492 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2019gitlab # [ 246.140879] gitaly[905]: time="2026-09-14T09:31:39.218Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:39.218" grpc.time_ms=0.279 pid=905 span.kind=server system=grpc2020gitlab # [ 246.141564] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.084Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b20dd5d2f3c85ca83d1faac1","created_at":"2026-09-14T09:31:38.866Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.867Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b20dd5d2f3c85ca83d1faac1: done: 0.013372 sec","job_status":"done","queue_duration_s":0.202944,"scheduling_latency_s":0.202959,"redis_calls":7,"redis_duration_s":0.001006,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013372,"completed_at":"2026-09-14T09:31:39.084Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2021gitlab # [ 246.192690] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.085Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"e561f98c411460959ee98daf","created_at":"2026-09-14T09:31:38.868Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.869Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e561f98c411460959ee98daf: start","job_status":"start","queue_duration_s":0.216111,"scheduling_latency_s":0.216138}2022gitlab # [ 246.193414] gitaly[905]: time="2026-09-14T09:31:39.242Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:39.242" grpc.time_ms=0.26 pid=905 span.kind=server system=grpc2023gitlab # [ 246.193927] gitaly[905]: time="2026-09-14T09:31:39.321Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:39.320" grpc.time_ms=0.361 pid=905 span.kind=server system=grpc2024gitlab # [ 246.199695] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.109Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"e561f98c411460959ee98daf","created_at":"2026-09-14T09:31:38.868Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.869Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e561f98c411460959ee98daf: done: 0.024399 sec","job_status":"done","queue_duration_s":0.216111,"scheduling_latency_s":0.216138,"redis_calls":7,"redis_duration_s":0.001355,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024399,"completed_at":"2026-09-14T09:31:39.109Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2025gitlab # [ 246.253383] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.111Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b331b5b9fec528a5746c1281","created_at":"2026-09-14T09:31:38.869Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.870Z","job_size_bytes":6,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b331b5b9fec528a5746c1281: start","job_status":"start","queue_duration_s":0.240514,"scheduling_latency_s":0.240532}2026gitlab # [ 246.254132] gitaly[905]: time="2026-09-14T09:31:39.325Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:09.189" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:39.190" operations="[repack_refs]" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027gitlab # [ 246.261678] gitaly[905]: time="2026-09-14T09:31:39.326Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:09.189" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:39.190" pid=905 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-14 09:30:40.934368313 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2028gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744'2029gitlab # [ 246.305291] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.120Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b331b5b9fec528a5746c1281","created_at":"2026-09-14T09:31:38.869Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.870Z","job_size_bytes":6,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b331b5b9fec528a5746c1281: done: 0.009638 sec","job_status":"done","queue_duration_s":0.240514,"scheduling_latency_s":0.240532,"redis_calls":7,"redis_duration_s":0.001057,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000311,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009638,"completed_at":"2026-09-14T09:31:39.120Z","load_balancing_strategy":"primary","db_duration_s":0.000439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 246.310280] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f054157aa515899877460307","created_at":"2026-09-14T09:31:38.871Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.872Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f054157aa515899877460307: start","job_status":"start","queue_duration_s":0.250152,"scheduling_latency_s":0.250184}2031gitlab # [ 246.324760] gitaly[905]: time="2026-09-14T09:31:39.331Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 diskcache=ae6df83c-d9ff-46cd-a6b0-2a2b4037e0cf grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:09.189" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:39.190" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032gitlab # [ 246.333764] gitaly[905]: time="2026-09-14T09:31:39.331Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=72 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7378 command.oublock=0 command.real_time_ms=136 command.spawn_token_fork_ms=0 command.system_time_ms=43 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:09.189" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:39.190" grpc.time_ms=140.943 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2033gitlab # [ 246.346180] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.140Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f054157aa515899877460307","created_at":"2026-09-14T09:31:38.871Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.872Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f054157aa515899877460307: done: 0.017884 sec","job_status":"done","queue_duration_s":0.250152,"scheduling_latency_s":0.250184,"redis_calls":7,"redis_duration_s":0.001062,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017884,"completed_at":"2026-09-14T09:31:39.140Z","load_balancing_strategy":"primary","db_duration_s":0.003085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 246.386137] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.141Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"2c3e1f48f94ed7fbd9cbb7b5","created_at":"2026-09-14T09:31:38.909Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.909Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2c3e1f48f94ed7fbd9cbb7b5: start","job_status":"start","queue_duration_s":0.23125,"scheduling_latency_s":0.231265}2035gitlab # [ 246.395946] gitaly[905]: time="2026-09-14T09:31:39.344Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:09.189" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:39.190" pid=905 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2036gitlab # [ 246.443694] gitaly[905]: time="2026-09-14T09:31:39.344Z" level=info msg="optimized repository" clean_stale_data_time=17.109717ms clean_work_trees_time="80.736µs" commit_graph_time="4.47µs" correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.749µs" pid=905 prune_time="3.073µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.749µs" storage_name=default system=housekeeping total_time=17.311418ms2037gitlab # [ 246.444501] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.399Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"2c3e1f48f94ed7fbd9cbb7b5","created_at":"2026-09-14T09:31:38.909Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-5c476aa40fbc44db","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:38.909Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2c3e1f48f94ed7fbd9cbb7b5: done: 0.25859 sec","job_status":"done","queue_duration_s":0.23125,"scheduling_latency_s":0.231265,"gitaly_calls":3,"gitaly_duration_s":0.147656,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008912,"redis_read_bytes":144,"redis_write_bytes":16717,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000366,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000614,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001947,"redis_queues_read_bytes":24,"redis_queues_write_bytes":11223,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.00285,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000822,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000429,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072359,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.232748016,"duration_s":0.25859,"completed_at":"2026-09-14T09:31:39.399Z","load_balancing_strategy":"primary","db_duration_s":0.032503,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2038gitlab # [ 246.460171] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.400Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f61a8ca7f5c0e299a31d318c","created_at":"2026-09-14T09:31:39.029Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-145a1719743747d5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.032Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f61a8ca7f5c0e299a31d318c: start","job_status":"start","queue_duration_s":0.368893,"scheduling_latency_s":0.368909}2039gitlab # [ 246.498817] gitaly[905]: time="2026-09-14T09:31:39.344Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:09.189" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:39.190" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 246.499658] gitaly[905]: time="2026-09-14T09:31:39.370Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:09.369" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:39.369" grpc.time_ms=0.691 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 246.500499] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.418Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"f61a8ca7f5c0e299a31d318c","created_at":"2026-09-14T09:31:39.029Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-145a1719743747d5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.032Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-f61a8ca7f5c0e299a31d318c: done: 0.017118 sec","job_status":"done","queue_duration_s":0.368893,"scheduling_latency_s":0.368909,"redis_calls":7,"redis_duration_s":0.001045,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017118,"completed_at":"2026-09-14T09:31:39.417Z","load_balancing_strategy":"primary","db_duration_s":0.002919,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2042gitlab # [ 246.539695] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.418Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"af0844660d67d029a124bff6","created_at":"2026-09-14T09:31:39.032Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-145a1719743747d5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.034Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-af0844660d67d029a124bff6: start","job_status":"start","queue_duration_s":0.384138,"scheduling_latency_s":0.384153}2043gitlab # [ 246.546779] gitaly[905]: time="2026-09-14T09:31:39.392Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40F2TGN5WXY7VWVXAMV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:09.391" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:39.391" grpc.time_ms=0.525 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044gitlab # [ 246.592673] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.458Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"af0844660d67d029a124bff6","created_at":"2026-09-14T09:31:39.032Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-145a1719743747d5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.034Z","job_size_bytes":65,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-af0844660d67d029a124bff6: done: 0.039076 sec","job_status":"done","queue_duration_s":0.384138,"scheduling_latency_s":0.384153,"redis_calls":10,"redis_duration_s":0.001686,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000819,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039076,"completed_at":"2026-09-14T09:31:39.458Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2045gitlab # [ 246.596524] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.459Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f41c0815f9e4e04a9f522363","created_at":"2026-09-14T09:31:39.088Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-145a1719743747d5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.090Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f41c0815f9e4e04a9f522363: start","job_status":"start","queue_duration_s":0.368374,"scheduling_latency_s":0.368396}2046gitlab # [ 246.611175] gitaly[905]: time="2026-09-14T09:31:39.510Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:09.509" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:39.510" grpc.time_ms=0.773 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 246.649716] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.546Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f41c0815f9e4e04a9f522363","created_at":"2026-09-14T09:31:39.088Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-145a1719743747d5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.090Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f41c0815f9e4e04a9f522363: done: 0.087314 sec","job_status":"done","queue_duration_s":0.368374,"scheduling_latency_s":0.368396,"gitaly_calls":2,"gitaly_duration_s":0.006019,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00504,"redis_read_bytes":80,"redis_write_bytes":5119,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000779,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000815,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000225,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00018,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000422,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062139,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.065855577,"duration_s":0.087314,"completed_at":"2026-09-14T09:31:39.546Z","load_balancing_strategy":"primary","db_duration_s":0.018446,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2048gitlab # [ 246.667989] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.547Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f3c813548914e319bae24ed7","created_at":"2026-09-14T09:31:39.185Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.186Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f3c813548914e319bae24ed7: start","job_status":"start","queue_duration_s":0.361292,"scheduling_latency_s":0.36131}2049gitlab # [ 246.710632] gitaly[905]: time="2026-09-14T09:31:39.536Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM40XR8ZN480PCKS9ZC3DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:09.535" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:39.536" grpc.time_ms=0.659 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2050gitlab # [ 246.711805] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.575Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f3c813548914e319bae24ed7","created_at":"2026-09-14T09:31:39.185Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.186Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f3c813548914e319bae24ed7: done: 0.027221 sec","job_status":"done","queue_duration_s":0.361292,"scheduling_latency_s":0.36131,"redis_calls":7,"redis_duration_s":0.0010860000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027221,"completed_at":"2026-09-14T09:31:39.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004791,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2051gitlab # [ 246.756730] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.576Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"239bc73f993af9d380365b04","created_at":"2026-09-14T09:31:39.336Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.338Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-239bc73f993af9d380365b04: start","job_status":"start","queue_duration_s":0.237878,"scheduling_latency_s":0.237919}2052gitlab # [ 246.757657] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.592Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"239bc73f993af9d380365b04","created_at":"2026-09-14T09:31:39.336Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.338Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-239bc73f993af9d380365b04: done: 0.016575 sec","job_status":"done","queue_duration_s":0.237878,"scheduling_latency_s":0.237919,"redis_calls":7,"redis_duration_s":0.001013,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016575,"completed_at":"2026-09-14T09:31:39.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002451,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053gitlab # [ 246.809333] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.593Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"06b91c93443d815bab158aa1","created_at":"2026-09-14T09:31:39.345Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.348Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-06b91c93443d815bab158aa1: start","job_status":"start","queue_duration_s":0.244903,"scheduling_latency_s":0.244919}2054gitlab # [ 246.810123] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.610Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"06b91c93443d815bab158aa1","created_at":"2026-09-14T09:31:39.345Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.348Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-06b91c93443d815bab158aa1: done: 0.017237 sec","job_status":"done","queue_duration_s":0.244903,"scheduling_latency_s":0.244919,"redis_calls":7,"redis_duration_s":0.0009989999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017237,"completed_at":"2026-09-14T09:31:39.610Z","load_balancing_strategy":"primary","db_duration_s":0.002788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055gitlab # [ 246.871263] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.611Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9977ef5ffd92c3ee9451c8c5","created_at":"2026-09-14T09:31:39.353Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.355Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9977ef5ffd92c3ee9451c8c5: start","job_status":"start","queue_duration_s":0.256802,"scheduling_latency_s":0.256818}2056gitlab # [ 246.872116] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.630Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9977ef5ffd92c3ee9451c8c5","created_at":"2026-09-14T09:31:39.353Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.355Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9977ef5ffd92c3ee9451c8c5: done: 0.018971 sec","job_status":"done","queue_duration_s":0.256802,"scheduling_latency_s":0.256818,"redis_calls":10,"redis_duration_s":0.001411,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000436,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018971,"completed_at":"2026-09-14T09:31:39.630Z","load_balancing_strategy":"primary","db_duration_s":0.002938,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057gitlab # [ 246.887166] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.631Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1da9cb4b84418a9551748f5f","created_at":"2026-09-14T09:31:39.355Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.356Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1da9cb4b84418a9551748f5f: start","job_status":"start","queue_duration_s":0.275465,"scheduling_latency_s":0.275483}2058gitlab # [ 246.938797] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.647Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1da9cb4b84418a9551748f5f","created_at":"2026-09-14T09:31:39.355Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.356Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1da9cb4b84418a9551748f5f: done: 0.016137 sec","job_status":"done","queue_duration_s":0.275465,"scheduling_latency_s":0.275483,"redis_calls":7,"redis_duration_s":0.001035,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016137,"completed_at":"2026-09-14T09:31:39.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059gitlab # [ 246.939919] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.648Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2ce03b8c1b05b39025595a0a","created_at":"2026-09-14T09:31:39.356Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.357Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2ce03b8c1b05b39025595a0a: start","job_status":"start","queue_duration_s":0.290933,"scheduling_latency_s":0.290948}2060gitlab # [ 246.985908] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.666Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"2ce03b8c1b05b39025595a0a","created_at":"2026-09-14T09:31:39.356Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.357Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-2ce03b8c1b05b39025595a0a: done: 0.01732 sec","job_status":"done","queue_duration_s":0.290933,"scheduling_latency_s":0.290948,"redis_calls":8,"redis_duration_s":0.002026,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001735,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01732,"completed_at":"2026-09-14T09:31:39.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061gitlab # [ 246.990949] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.667Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fbdc382e3a035c2ca928a8f5","created_at":"2026-09-14T09:31:39.358Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.359Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fbdc382e3a035c2ca928a8f5: start","job_status":"start","queue_duration_s":0.307672,"scheduling_latency_s":0.307689}2062gitlab # [ 247.032044] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.681Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fbdc382e3a035c2ca928a8f5","created_at":"2026-09-14T09:31:39.358Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.359Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fbdc382e3a035c2ca928a8f5: done: 0.014144 sec","job_status":"done","queue_duration_s":0.307672,"scheduling_latency_s":0.307689,"redis_calls":7,"redis_duration_s":0.000985,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014144,"completed_at":"2026-09-14T09:31:39.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001783,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 247.045095] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.682Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"58748b4404c7107e386b49e4","created_at":"2026-09-14T09:31:39.359Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.360Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-58748b4404c7107e386b49e4: start","job_status":"start","queue_duration_s":0.321599,"scheduling_latency_s":0.321613}2064gitlab # [ 247.086110] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.698Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"58748b4404c7107e386b49e4","created_at":"2026-09-14T09:31:39.359Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.360Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-58748b4404c7107e386b49e4: done: 0.016678 sec","job_status":"done","queue_duration_s":0.321599,"scheduling_latency_s":0.321613,"redis_calls":7,"redis_duration_s":0.001054,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016678,"completed_at":"2026-09-14T09:31:39.698Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 247.097665] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8c98bafdb809c6def82ad83e","created_at":"2026-09-14T09:31:39.360Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.361Z","job_size_bytes":6,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8c98bafdb809c6def82ad83e: start","job_status":"start","queue_duration_s":0.338111,"scheduling_latency_s":0.338126}2066gitlab # [ 247.137723] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.709Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8c98bafdb809c6def82ad83e","created_at":"2026-09-14T09:31:39.360Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.361Z","job_size_bytes":6,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8c98bafdb809c6def82ad83e: done: 0.009089 sec","job_status":"done","queue_duration_s":0.338111,"scheduling_latency_s":0.338126,"redis_calls":7,"redis_duration_s":0.0010040000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009089,"completed_at":"2026-09-14T09:31:39.708Z","load_balancing_strategy":"primary","db_duration_s":0.000317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2067gitlab # [ 247.183808] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.709Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a0259bbefc1497bc2287d1b6","created_at":"2026-09-14T09:31:39.362Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.362Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a0259bbefc1497bc2287d1b6: start","job_status":"start","queue_duration_s":0.346975,"scheduling_latency_s":0.34699}2068gitlab # [ 247.184638] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.725Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a0259bbefc1497bc2287d1b6","created_at":"2026-09-14T09:31:39.362Z","trace_propagation_headers":{"sentry-trace":"423de50f390d405c8613552b396dc626-bd051947c39447ea","baggage":"sentry-trace_id=423de50f390d405c8613552b396dc626,sentry-release=Unknown"},"correlation_id":"01M2FM40F2TGN5WXY7VWVXAMV4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.362Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a0259bbefc1497bc2287d1b6: done: 0.01519 sec","job_status":"done","queue_duration_s":0.346975,"scheduling_latency_s":0.34699,"redis_calls":7,"redis_duration_s":0.000985,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01519,"completed_at":"2026-09-14T09:31:39.725Z","load_balancing_strategy":"primary","db_duration_s":0.002508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2069gitlab # [ 247.225502] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.726Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3c459ebd1966e5e820b2f0f3","created_at":"2026-09-14T09:31:39.498Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-b16766f6298e48e5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.500Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3c459ebd1966e5e820b2f0f3: start","job_status":"start","queue_duration_s":0.225752,"scheduling_latency_s":0.225768}2070gitlab # [ 247.226336] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:39.747Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"3c459ebd1966e5e820b2f0f3","created_at":"2026-09-14T09:31:39.498Z","trace_propagation_headers":{"sentry-trace":"5ef53a62e97a4504847ebc2aab2a7ee6-b16766f6298e48e5","baggage":"sentry-trace_id=5ef53a62e97a4504847ebc2aab2a7ee6,sentry-release=Unknown"},"correlation_id":"01M2FM40XR8ZN480PCKS9ZC3DS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:39.500Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-3c459ebd1966e5e820b2f0f3: done: 0.021558 sec","job_status":"done","queue_duration_s":0.225752,"scheduling_latency_s":0.225768,"redis_calls":7,"redis_duration_s":0.001075,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021558,"completed_at":"2026-09-14T09:31:39.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 247.301340] nixbot[2634]: INFO: ::ffff:127.0.0.1:56202 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 200 OK2072gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a4867eecf06d5de25f006944960c955ecd839744', in 0.98 seconds)2073(finished: subtest: push delivers the webhook, in 48.76 seconds)2074subtest: eval and build post commit statuses2075gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses2076gitlab # [ 247.341340] gitaly[905]: time="2026-09-14T09:31:40.623Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM40ZXJM7S786YEMXA6G98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:40.617" grpc.time_ms=6.646 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2077gitlab # [ 247.341963] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 17472078gitlab # [ 247.364799] nix-daemon[2793]: accepted connection from pid 3622, user nixbot2079gitlab # [ 247.393938] gitaly[905]: time="2026-09-14T09:31:40.676Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM42PTG9VES87M5Y6RFFVP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:10.674" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:40.675" grpc.time_ms=0.871 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2080gitlab # [ 247.411906] gitaly[905]: time="2026-09-14T09:31:40.694Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM42PJMYBKX5SGDYDV8VZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:40.676" pid=905 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2081gitlab # [ 247.412636] gitaly[905]: time="2026-09-14T09:31:40.694Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=10 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM42PJMYBKX5SGDYDV8VZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:40.676" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2082gitlab # [ 247.425840] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:40 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 732083gitlab # [ 247.434804] gitaly[905]: time="2026-09-14T09:31:40.717Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=10 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM42PJMYBKX5SGDYDV8VZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T09:31:40.676" grpc.time_ms=18.148 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc2084gitlab # [ 247.435503] gitlab-workhorse[832]: time="2026-09-14T09:31:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672085gitlab # [ 247.435921] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 912086gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses, in 0.13 seconds)2087gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses2088gitlab # [ 248.559500] gitaly[905]: time="2026-09-14T09:31:41.841Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM43TABANPYGEMJTCNAMZ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:11.839" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:41.840" grpc.time_ms=0.925 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2089gitlab # [ 248.612740] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:41 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1242090gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses, in 0.17 seconds)2091gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses2092gitlab # [ 249.712930] gitaly[905]: time="2026-09-14T09:31:42.995Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM44YVX8R343Z0V9XEW013 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:12.193" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:42.993" grpc.time_ms=1.016 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093gitlab # [ 249.741371] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 842094gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses, in 0.13 seconds)2095gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses2096gitlab # [ 250.842294] gitaly[905]: time="2026-09-14T09:31:44.124Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM4623M5ZYPR6G9K6TP4CN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:14.122" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:44.123" grpc.time_ms=1.033 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097gitlab # [ 250.868740] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 832098gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses, in 0.13 seconds)2099gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses2100gitlab # [ 251.967478] gitaly[905]: time="2026-09-14T09:31:45.249Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM475CNW3BBDRX3HB411KX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:15.247" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:45.248" grpc.time_ms=0.958 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2101gitlab # [ 251.999246] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 842102gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses, in 0.13 seconds)2103gitlab # [ 252.302587] nix-daemon[2793]: accepted connection from pid 3682, user nixbot2104gitlab # [ 252.316186] nix-daemon[2793]: accepted connection from pid 3622, user nixbot2105gitlab # [ 252.398444] gitaly[905]: time="2026-09-14T09:31:45.681Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:15.679" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:45.680" grpc.time_ms=0.905 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106gitlab # [ 252.464959] gitaly[905]: time="2026-09-14T09:31:45.747Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:55.175" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:45.735" grpc.time_ms=11.677 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2107gitlab # [ 252.578218] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:45.859Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"217abb3c836172d6c6a57d21","created_at":"2026-09-14T09:31:45.851Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-cbd6d9841cfe4dcb","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:45.857Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-217abb3c836172d6c6a57d21: start","job_status":"start","queue_duration_s":0.002197,"scheduling_latency_s":0.002252}2108gitlab # [ 252.629365] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2702109gitlab # [ 252.631778] nixbot[2634]: {"ts": "2026-09-14T09:31:45.913Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 \"HTTP/1.1 201 Created\""}2110gitlab # [ 252.691589] gitaly[905]: time="2026-09-14T09:31:45.973Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM47VWZNW8KDZ9GHDG7SZF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:15.171" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:45.972" grpc.time_ms=0.859 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2111gitlab # [ 252.727369] gitaly[905]: time="2026-09-14T09:31:46.010Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:46.009" grpc.time_ms=0.633 pid=905 span.kind=server system=grpc2112gitlab # [ 252.746668] gitaly[905]: time="2026-09-14T09:31:46.029Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=18 command.system_time_ms=15 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2FM47VWZNW8KDZ9GHDG7SZF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:56.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:46.006" grpc.time_ms=22.774 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab # [ 252.766851] gitaly[905]: time="2026-09-14T09:31:46.049Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:46.049" grpc.time_ms=0.326 pid=905 span.kind=server system=grpc2114gitlab # [ 252.793332] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1592115gitlab # [ 252.794745] nixbot[2634]: {"ts": "2026-09-14T09:31:46.077Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 \"HTTP/1.1 201 Created\""}2116gitlab # [ 252.803710] gitaly[905]: time="2026-09-14T09:31:46.086Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3893 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T09:31:46.086" grpc.time_ms=0.419 pid=905 span.kind=server system=grpc2117gitlab # [ 252.819349] gitaly[905]: time="2026-09-14T09:31:46.102Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG diskcache=10d26cd2-d69e-4cc7-a123-0c5f9dbc0e5d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:15.046" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:45.947" pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2118gitlab # [ 252.819597] gitaly[905]: time="2026-09-14T09:31:46.102Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=117 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7244 command.oublock=0 command.real_time_ms=148 command.spawn_token_fork_ms=0 command.system_time_ms=86 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T09:32:15.046" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:45.947" grpc.time_ms=154.916 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2119gitlab # [ 252.859210] gitaly[905]: time="2026-09-14T09:31:46.141Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM481JGTC4QDH4Q5CED82V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:16.140" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:46.141" grpc.time_ms=0.61 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2120gitlab # [ 252.870947] gitaly[905]: time="2026-09-14T09:31:46.153Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T09:32:16.152" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:46.152" grpc.time_ms=1.156 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2121gitlab # [ 252.893140] gitaly[905]: time="2026-09-14T09:31:46.176Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FM481JGTC4QDH4Q5CED82V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T09:31:56.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T09:31:46.170" grpc.time_ms=5.88 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2122gitlab # [ 252.905148] gitaly[905]: time="2026-09-14T09:31:46.177Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM47K9MCJGRB10AMBVW6JG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:16.176" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:46.176" grpc.time_ms=0.501 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2123gitlab # [ 252.905453] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.184Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1e40da5687e2a9ec2eaedaac","created_at":"2026-09-14T09:31:46.182Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"217abb3c836172d6c6a57d21","job_size_bytes":3,"pid":2506,"job_status":"deduplicated","message":"PipelineProcessWorker JID-1e40da5687e2a9ec2eaedaac: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2124gitlab # [ 252.906093] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.185Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"217abb3c836172d6c6a57d21","created_at":"2026-09-14T09:31:45.851Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-cbd6d9841cfe4dcb","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:45.857Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2506,"job_status":"rescheduled","message":"PipelineProcessWorker JID-217abb3c836172d6c6a57d21: rescheduled"}2125gitlab # [ 252.907710] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.190Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"217abb3c836172d6c6a57d21","created_at":"2026-09-14T09:31:45.851Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-cbd6d9841cfe4dcb","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:45.857Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-217abb3c836172d6c6a57d21: done: 0.331098 sec","job_status":"done","queue_duration_s":0.002197,"scheduling_latency_s":0.002252,"gitaly_calls":3,"gitaly_duration_s":0.16366,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013786,"redis_read_bytes":239,"redis_write_bytes":21779,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001257,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001151,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.00305,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14464,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.004641,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000358,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000591,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001481,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124092,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.295236406,"duration_s":0.331098,"completed_at":"2026-09-14T09:31:46.190Z","load_balancing_strategy":"primary","db_duration_s":0.033713,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2126gitlab # [ 252.987492] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"374c56bb2ae66ad79fa92de9","created_at":"2026-09-14T09:31:45.936Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:45.939Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-374c56bb2ae66ad79fa92de9: start","job_status":"start","queue_duration_s":0.330417,"scheduling_latency_s":0.33046}2127gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses2128gitlab # [ 253.024094] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2072129gitlab # [ 253.026357] nixbot[2634]: {"ts": "2026-09-14T09:31:46.309Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a4867eecf06d5de25f006944960c955ecd839744 \"HTTP/1.1 201 Created\""}2130gitlab # [ 253.045470] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.328Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"374c56bb2ae66ad79fa92de9","created_at":"2026-09-14T09:31:45.936Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:45.939Z","job_size_bytes":23,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-374c56bb2ae66ad79fa92de9: done: 0.058165 sec","job_status":"done","queue_duration_s":0.330417,"scheduling_latency_s":0.33046,"redis_calls":7,"redis_duration_s":0.001431,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058165,"completed_at":"2026-09-14T09:31:46.327Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01402,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2131gitlab # [ 253.066863] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.329Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a7656fad1e8772545239504a","created_at":"2026-09-14T09:31:46.108Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.110Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a7656fad1e8772545239504a: start","job_status":"start","queue_duration_s":0.218613,"scheduling_latency_s":0.218632}2132gitlab # [ 253.081169] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.363Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a7656fad1e8772545239504a","created_at":"2026-09-14T09:31:46.108Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.110Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a7656fad1e8772545239504a: done: 0.033729 sec","job_status":"done","queue_duration_s":0.218613,"scheduling_latency_s":0.218632,"redis_calls":7,"redis_duration_s":0.0028309999999999997,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002247,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000584,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033729,"completed_at":"2026-09-14T09:31:46.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2133gitlab # [ 253.130249] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.369Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7f38c4eff58de3cd9529b11c","created_at":"2026-09-14T09:31:46.115Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.118Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7f38c4eff58de3cd9529b11c: start","job_status":"start","queue_duration_s":0.250548,"scheduling_latency_s":0.25057}2134gitlab # [ 253.130865] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.388Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7f38c4eff58de3cd9529b11c","created_at":"2026-09-14T09:31:46.115Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.118Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7f38c4eff58de3cd9529b11c: done: 0.019536 sec","job_status":"done","queue_duration_s":0.250548,"scheduling_latency_s":0.25057,"redis_calls":7,"redis_duration_s":0.002018,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000329,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019536,"completed_at":"2026-09-14T09:31:46.388Z","load_balancing_strategy":"primary","db_duration_s":0.00313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135gitlab # [ 253.181262] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"7ac968d82dcd5cbf38823426","created_at":"2026-09-14T09:31:46.125Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.128Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7ac968d82dcd5cbf38823426: start","job_status":"start","queue_duration_s":0.261289,"scheduling_latency_s":0.261304}2136gitlab # [ 253.181967] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.407Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"7ac968d82dcd5cbf38823426","created_at":"2026-09-14T09:31:46.125Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.128Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7ac968d82dcd5cbf38823426: done: 0.018067 sec","job_status":"done","queue_duration_s":0.261289,"scheduling_latency_s":0.261304,"redis_calls":10,"redis_duration_s":0.0013809999999999998,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000384,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018067,"completed_at":"2026-09-14T09:31:46.407Z","load_balancing_strategy":"primary","db_duration_s":0.002943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2137gitlab # [ 253.229323] gitaly[905]: time="2026-09-14T09:31:46.428Z" level=info msg="finished unary call with code OK" catfile.duration_ms=62 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=62 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FM488M0N29GXYTWQ9Y8W43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T09:32:16.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T09:31:46.364" grpc.time_ms=63.186 pid=905 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2138gitlab # [ 253.229938] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.408Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ac9ad08fcbc88a407bcfe1fb","created_at":"2026-09-14T09:31:46.128Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.130Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ac9ad08fcbc88a407bcfe1fb: start","job_status":"start","queue_duration_s":0.278674,"scheduling_latency_s":0.278688}2139gitlab # [ 253.230518] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ac9ad08fcbc88a407bcfe1fb","created_at":"2026-09-14T09:31:46.128Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.130Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ac9ad08fcbc88a407bcfe1fb: done: 0.073543 sec","job_status":"done","queue_duration_s":0.278674,"scheduling_latency_s":0.278688,"redis_calls":7,"redis_duration_s":0.004449000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.073543,"completed_at":"2026-09-14T09:31:46.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006771,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses, in 0.30 seconds)2141(finished: subtest: eval and build post commit statuses, in 5.98 seconds)2142(finished: run the VM test script, in 303.39 seconds)2143gitlab # [ 253.264795] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a61c3564ba6a412f09acf212","created_at":"2026-09-14T09:31:46.130Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.131Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a61c3564ba6a412f09acf212: start","job_status":"start","queue_duration_s":0.351563,"scheduling_latency_s":0.351584}2144gitlab # [ 253.272079] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.498Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a61c3564ba6a412f09acf212","created_at":"2026-09-14T09:31:46.130Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.131Z","job_size_bytes":19,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a61c3564ba6a412f09acf212: done: 0.015245 sec","job_status":"done","queue_duration_s":0.351563,"scheduling_latency_s":0.351584,"redis_calls":8,"redis_duration_s":0.001066,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015245,"completed_at":"2026-09-14T09:31:46.498Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 253.312758] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/14:09:31:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a4867eecf06d5de25f006944960c955ecd839744/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 2042146gitlab # [ 253.313548] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.499Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e0989049345aa0985b5e23d9","created_at":"2026-09-14T09:31:46.132Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.133Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e0989049345aa0985b5e23d9: start","job_status":"start","queue_duration_s":0.365995,"scheduling_latency_s":0.366009}2147gitlab # [ 253.325590] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.516Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e0989049345aa0985b5e23d9","created_at":"2026-09-14T09:31:46.132Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.133Z","job_size_bytes":66,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e0989049345aa0985b5e23d9: done: 0.016912 sec","job_status":"done","queue_duration_s":0.365995,"scheduling_latency_s":0.366009,"redis_calls":7,"redis_duration_s":0.004693,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016912,"completed_at":"2026-09-14T09:31:46.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001489,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 253.371890] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.517Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a432acb3b65c5d4fbff88f8d","created_at":"2026-09-14T09:31:46.134Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.135Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a432acb3b65c5d4fbff88f8d: start","job_status":"start","queue_duration_s":0.382409,"scheduling_latency_s":0.382423}2149gitlab # [ 253.412345] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.599Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a432acb3b65c5d4fbff88f8d","created_at":"2026-09-14T09:31:46.134Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-14T09:31:46.135Z","job_size_bytes":3,"pid":2506,"sidekiq_tid":"m7xa","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a432acb3b65c5d4fbff88f8d: done: 0.082485 sec","job_status":"done","queue_duration_s":0.382409,"scheduling_latency_s":0.382423,"redis_calls":7,"redis_duration_s":0.003941,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082485,"completed_at":"2026-09-14T09:31:46.599Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.058193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150test script finished in 303.54s2151cleanup2152kill QemuMachine (pid 46)2153gitlab # [ 253.417614] sidekiq-cluster[2506]: {"severity":"INFO","time":"2026-09-14T09:31:46.602Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"4368a3ba37a8b8cbbf792e0e","created_at":"2026-09-14T09:31:46.138Z","trace_propagation_headers":{"sentry-trace":"003ce4a9937b4139adaefd50fc88b9b8-df0762a41dca480c","baggage":"sentry-trace_id=003ce4a9937b4139adaefd50fc88b9b8,sentry-release=Unknown"},"correlation_id":"01M2FM47K9MCJGRB10AMBVW6JG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2154gitlab # p2155(finished: cleanup, in 1.38 seconds)2156additionally exposed symbols:2157 gitlab,2158 vlan1,2159 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2160[]2161[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-14T09:30:50.859Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2162[]2163[]2164[]2165[]2166[]2167[]2168[]2169[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-14T09:31:38.799123Z', 'started_at': '2026-09-14T09:31:39.109294Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2170[{'id': 5, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T09:31:39.082Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2171[{'id': 5, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T09:31:39.082Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2172[{'id': 5, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T09:31:39.082Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2173[{'id': 5, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T09:31:39.082Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2174[{'id': 5, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T09:31:39.082Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2175[{'id': 5, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 6s (1 warning)', 'created_at': '2026-09-14T09:31:39.082Z', 'started_at': None, 'finished_at': '2026-09-14T09:31:45.840Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'a4867eecf06d5de25f006944960c955ecd839744', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-14T09:31:46.056Z', 'started_at': None, 'finished_at': '2026-09-14T09:31:46.274Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]