vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #352
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.j0SRtuoKoo', fmt=raw size=107374182412(finished: start all VMs, in 48.06 seconds)13gitlab # mke2fs 1.47.4 (6-Mar-2025)14gitlab: waiting for unit setup-gitlab.service15gitlab # Discarding device blocks: 0/262144 done16gitlab: waiting for the VM to finish booting17gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes18gitlab # Filesystem UUID: ba6695a6-135b-4022-9d85-09d574868ac519gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wh90yqx9w6s2lqyv6pbi62j6c4361bxx-nixos-system-gitlab-test/init regInfo=/nix/store/bwaz89p7kvj0j3i4h5nczrc1awld15nk-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 463350975 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000004] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000874] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002734] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002744] Using GB pages for direct mapping73gitlab # [ 0.002869] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002877] ACPI: Early table checksum verification disabled75gitlab # [ 0.002880] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002883] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002887] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002893] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002895] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002897] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002898] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002900] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002901] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002902] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002903] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002903] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002904] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003165] No NUMA configuration found90gitlab # [ 0.003166] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003168] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005285] Zone ranges:93gitlab # [ 0.005285] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005287] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005288] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005289] Device empty97gitlab # [ 0.005290] Movable zone start for each node98gitlab # [ 0.005290] Early memory node ranges99gitlab # [ 0.005291] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005292] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005293] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005294] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005315] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005560] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.106150] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.106571] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.106587] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.106618] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.106621] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.106622] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.106623] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.106624] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.106625] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.106628] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.106629] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.106633] TSC deadline timer available117gitlab # [ 0.106638] CPU topo: Max. logical packages: 1118gitlab # [ 0.106639] CPU topo: Max. logical dies: 1119gitlab # [ 0.106639] CPU topo: Max. dies per package: 1120gitlab # [ 0.106643] CPU topo: Max. threads per core: 1121gitlab # [ 0.106643] CPU topo: Num. cores per package: 4122gitlab # [ 0.106644] CPU topo: Num. threads per package: 4123gitlab # [ 0.106644] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.106667] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.106683] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.106686] kvm-guest: setup PV sched yield127gitlab # [ 0.106706] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.106708] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.106709] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.106710] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.106712] Booting paravirtualized kernel on KVM132gitlab # [ 0.106715] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.111160] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.113243] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.113290] kvm-guest: PV spinlocks enabled136gitlab # [ 0.113292] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.113295] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wh90yqx9w6s2lqyv6pbi62j6c4361bxx-nixos-system-gitlab-test/init regInfo=/nix/store/bwaz89p7kvj0j3i4h5nczrc1awld15nk-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.113388] Unknown kernel command line parameters "regInfo=/nix/store/bwaz89p7kvj0j3i4h5nczrc1awld15nk-closure-info/registration", will be passed to user space.139gitlab # [ 0.113401] random: crng init done140gitlab # [ 0.113401] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.121609] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.125807] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.125878] software IO TLB: area num 4.144gitlab # [ 0.194886] Fallback order for Node 0: 0145gitlab # [ 0.194893] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.194895] Policy zone: Normal147gitlab # [ 0.197250] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.206455] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.219193] allocated 12582912 bytes of page_ext150gitlab # [ 0.228993] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.228995] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.229849] Dynamic Preempt: lazy153gitlab # [ 0.230007] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.230008] rcu: RCU event tracing is enabled.155gitlab # [ 0.230009] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.230010] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.230011] Rude variant of Tasks RCU enabled.158gitlab # [ 0.230011] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.230012] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.230012] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.230041] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.230042] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.230043] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.234372] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.234682] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.234690] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.234810] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.238409] Console: colour VGA+ 80x25169gitlab # [ 0.238412] printk: legacy console [tty0] enabled170gitlab # [ 0.268676] printk: legacy console [ttyS0] enabled171gitlab # [ 0.376758] ACPI: Core revision 20250807172gitlab # [ 0.377708] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.379308] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.380359] x2apic enabled175gitlab # [ 0.381119] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.382105] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.383368] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.384872] kvm-guest: setup PV IPIs179gitlab # [ 0.386503] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.387567] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.389406] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.390489] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.391523] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.393404] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.394414] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.395405] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.396404] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.398201] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.399172] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.400220] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.401412] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.402405] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.403404] active return thunk: srso_alias_return_thunk194gitlab # [ 0.404424] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.406404] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.407404] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.408403] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.409404] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.410404] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.411404] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.412404] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.414404] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.415404] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.416404] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.417403] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.418403] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.448879] Freeing SMP alternatives memory: 44K208gitlab # [ 0.449407] pid_max: default: 32768 minimum: 301209gitlab # [ 0.450510] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.451502] landlock: Up and running.211gitlab # [ 0.452404] Yama: becoming mindful.212gitlab # [ 0.453254] LSM support for eBPF active213gitlab # [ 0.454101] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.455532] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.458179] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.458949] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.459410] ... version: 2218gitlab # [ 0.460162] ... bit width: 48219gitlab # [ 0.460407] ... generic counters: 6220gitlab # [ 0.461185] ... generic bitmap: 000000000000003f221gitlab # [ 0.461417] ... fixed-purpose counters: 0222gitlab # [ 0.462193] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.462406] ... value mask: 0000ffffffffffff224gitlab # [ 0.463366] ... max period: 00007fffffffffff225gitlab # [ 0.464144] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.464538] signal: max sigframe size: 3376227gitlab # [ 0.465390] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.466066] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.466593] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.471863] smp: Bringing up secondary CPUs ...231gitlab # [ 0.472983] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.473414] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.476584] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.478212] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.479574] Memory: 6011304K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved)236gitlab # [ 0.481407] devtmpfs: initialized237gitlab # [ 0.482535] x86/mm: Memory block size: 128MB238gitlab # [ 0.485575] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.487511] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.488525] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.489697] PM: RTC time: 16:30:27, date: 2026-09-06242gitlab # [ 0.493120] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.494687] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.495463] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.497444] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.498417] audit: initializing netlink subsys (disabled)247gitlab # [ 0.499614] audit: type=2000 audit(1788712228.698:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.499699] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.501408] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.502406] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.503406] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.504406] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.505438] cpuidle: using governor menu254gitlab # [ 0.507621] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.508629] PCI: Using configuration type 1 for base access256gitlab # [ 0.509407] PCI: Using configuration type 1 for extended access257gitlab # [ 0.510565] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.516702] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.519413] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.521414] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.524413] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.532472] ACPI: Added _OSI(Module Device)263gitlab # [ 0.533415] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.535415] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.542308] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.550089] ACPI: Interpreter enabled267gitlab # [ 0.551480] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.553417] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.555480] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.559415] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.561789] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.574421] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.576421] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.579140] acpiphp: Slot [3] registered275gitlab # [ 0.580441] acpiphp: Slot [4] registered276gitlab # [ 0.582452] acpiphp: Slot [5] registered277gitlab # [ 0.583450] acpiphp: Slot [6] registered278gitlab # [ 0.585468] acpiphp: Slot [7] registered279gitlab # [ 0.586450] acpiphp: Slot [8] registered280gitlab # [ 0.587450] acpiphp: Slot [9] registered281gitlab # [ 0.588427] acpiphp: Slot [10] registered282gitlab # [ 0.589159] acpiphp: Slot [11] registered283gitlab # [ 0.590429] acpiphp: Slot [12] registered284gitlab # [ 0.591184] acpiphp: Slot [13] registered285gitlab # [ 0.591447] acpiphp: Slot [14] registered286gitlab # [ 0.592169] acpiphp: Slot [15] registered287gitlab # [ 0.593426] acpiphp: Slot [16] registered288gitlab # [ 0.594196] acpiphp: Slot [17] registered289gitlab # [ 0.594437] acpiphp: Slot [18] registered290gitlab # [ 0.595164] acpiphp: Slot [19] registered291gitlab # [ 0.596428] acpiphp: Slot [20] registered292gitlab # [ 0.597180] acpiphp: Slot [21] registered293gitlab # [ 0.597436] acpiphp: Slot [22] registered294gitlab # [ 0.598197] acpiphp: Slot [23] registered295gitlab # [ 0.599443] acpiphp: Slot [24] registered296gitlab # [ 0.600230] acpiphp: Slot [25] registered297gitlab # [ 0.601024] acpiphp: Slot [26] registered298gitlab # [ 0.601427] acpiphp: Slot [27] registered299gitlab # [ 0.602200] acpiphp: Slot [28] registered300gitlab # [ 0.603435] acpiphp: Slot [29] registered301gitlab # [ 0.604194] acpiphp: Slot [30] registered302gitlab # [ 0.604427] acpiphp: Slot [31] registered303gitlab # [ 0.605182] PCI host bridge to bus 0000:00304gitlab # [ 0.606413] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.607407] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.608408] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.609410] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.611408] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.612419] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.614321] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.620056] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.624589] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.631501] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.633481] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.636412] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.638413] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.641413] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.644877] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.650610] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.656043] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.660109] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.663428] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.665870] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.684496] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.685478] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.686440] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.687889] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.690683] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.696448] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.697379] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.698221] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.711407] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.713960] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.717443] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.718372] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.720211] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.722649] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.727644] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.729103] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.730429] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.732715] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.736443] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.737334] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.738201] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.741611] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.745437] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.746360] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.747222] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.750252] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.754480] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.755421] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.756429] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.764299] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.775535] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.778434] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.780468] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.784447] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.788883] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.797703] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.800469] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.806605] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.814457] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.816432] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.818456] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.825340] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.834435] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.836428] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.838466] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.851510] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.853803] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.856754] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.859752] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.862605] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.868826] iommu: Default domain type: Translated375gitlab # [ 0.870413] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.873830] ACPI: bus type USB registered377gitlab # [ 0.874491] usbcore: registered new interface driver usbfs378gitlab # [ 0.875425] usbcore: registered new interface driver hub379gitlab # [ 0.876365] usbcore: registered new device driver usb380gitlab # [ 0.878705] NetLabel: Initializing381gitlab # [ 0.879337] NetLabel: domain hash size = 128382gitlab # [ 0.880009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.881510] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.882419] PCI: Using ACPI for IRQ routing385gitlab # [ 0.883648] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.884403] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.884403] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.886412] vgaarb: loaded389gitlab # [ 0.887200] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.888064] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.893499] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.898921] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.899649] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.901400] pnp: PnP ACPI init395gitlab # [ 0.902663] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.912889] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.914505] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.915549] NET: Registered PF_INET protocol family399gitlab # [ 0.917830] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.936355] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.937949] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.939380] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.943222] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.944618] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.945862] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.947195] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.948387] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.949862] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.950883] NET: Registered PF_XDP protocol family410gitlab # [ 0.951774] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.952823] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.953896] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.955027] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.956168] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.957550] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.958568] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.961090] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.963899] PCI: CLS 0 bytes, default 64419gitlab # [ 0.964685] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.965266] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.965819] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.965965] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 0.990072] Initialise system trusted keyrings424gitlab # [ 0.991240] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 1.002702] Key type asymmetric registered426gitlab # [ 1.003467] Asymmetric key parser 'x509' registered427gitlab # [ 1.004383] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 1.006111] io scheduler mq-deadline registered429gitlab # [ 1.006924] io scheduler kyber registered430gitlab # [ 1.012285] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 1.013665] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 1.016228] Linux agpgart interface v0.103433gitlab # [ 1.017034] ACPI: bus type drm_connector registered434gitlab # [ 1.020331] usbcore: registered new interface driver usbserial_generic435gitlab # [ 1.021487] usbserial: USB Serial support registered for generic436gitlab # [ 1.022530] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 1.022530] Please enable it if your BIOS has the CPPC option.438gitlab # [ 1.024866] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 1.026352] drop_monitor: Initializing network drop monitor service440gitlab # [ 1.027659] NET: Registered PF_INET6 protocol family441gitlab # [ 1.029509] Segment Routing with IPv6442gitlab # [ 1.030234] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.033520] IPI shorthand broadcast: enabled444gitlab # [ 1.037999] sched_clock: Marking stable (891013442, 146713696)->(1063007215, -25280077)445gitlab # [ 1.039965] registered taskstats version 1446gitlab # [ 1.041034] Loading compiled-in X.509 certificates447gitlab # [ 1.050865] Demotion targets for Node 0: null448gitlab # [ 1.051990] Key type .fscrypt registered449gitlab # [ 1.052734] Key type fscrypt-provisioning registered450gitlab # [ 1.053784] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.054750] ima: Allocated hash algorithm: sha1452gitlab # [ 1.055610] ima: No architecture policies found453gitlab # [ 1.056785] PM: Magic number: 10:643:539454gitlab # [ 1.058574] RAS: Correctable Errors collector initialized.455gitlab # [ 1.063107] clk: Disabling unused clocks456gitlab # [ 1.063833] PM: genpd: Disabling unused power domains457gitlab # [ 1.285344] Freeing initrd memory: 29392K458gitlab # [ 1.293223] Freeing unused decrypted memory: 2028K459gitlab # [ 1.300168] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.303157] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.308488] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.315553] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.384849] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.385986] Run /init as init process465gitlab # [ 1.398409] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.419883] fuse: init (API version 7.45)467gitlab # [ 1.428291] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.436998] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.443318] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.511567] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.847802] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.887403] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.889692] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.891895] systemd[1]: Running in initrd.475gitlab # [ 1.894608] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.897693] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 2.011145] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 2.051586] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 2.062509] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 2.063759] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 2.065174] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 2.066351] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 2.067489] systemd[1]: Reached target Path Units.484gitlab # [ 2.068419] systemd[1]: Reached target Slice Units.485gitlab # [ 2.069338] systemd[1]: Reached target Swaps.486gitlab # [ 2.070177] systemd[1]: Reached target Timer Units.487gitlab # [ 2.071181] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 2.072627] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 2.073836] systemd[1]: Listening on Journal Sockets.490gitlab # [ 2.074854] systemd[1]: Listening on udev Control Socket.491gitlab # [ 2.075895] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 2.076863] systemd[1]: Reached target Socket Units.493gitlab # [ 2.078890] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 2.081111] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 2.083303] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 2.086252] systemd[1]: Starting Journal Service...497gitlab # [ 2.088153] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 2.090819] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 2.093383] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 2.097793] systemd[1]: Finished Create List of Static Device Nodes.501gitlab # [ 2.103886] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...502gitlab # [ 2.105616] systemd[1]: modprobe@configfs.service: Deactivated successfully.503gitlab # [ 2.114187] systemd[1]: Finished Load Kernel Module configfs.504gitlab # [ 2.115633] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config505gitlab # [ 2.119337] systemd-journald[91]: Collecting audit messages is disabled.506gitlab # [ 2.132859] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.507gitlab # [ 2.135815] systemd[1]: Starting Create Static Device Nodes in /dev...508gitlab # [ 2.150292] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.509gitlab # [ 2.150613] netfs: FS-Cache loaded510gitlab # [ 2.151511] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev511gitlab # [ 2.154699] systemd[1]: Finished Create Static Device Nodes in /dev.512gitlab # [ 2.156113] systemd[1]: Reached target Preparation for Local File Systems.513gitlab # [ 2.158639] systemd[1]: Reached target Local File Systems.514gitlab # [ 2.161096] 9pnet: Installing 9P2000 support515gitlab # [ 2.162941] systemd[1]: Starting Rule-based Manager for Device Events and Files...516gitlab # [ 2.165050] systemd[1]: Finished Load Kernel Modules.517gitlab # [ 2.168250] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.518gitlab # [ 2.174147] systemd[1]: Finished Load Kernel Module 9pnet_virtio.519gitlab # [ 2.175499] systemd[1]: Started Journal Service.520gitlab # [ 2.028823] systemd-modules-load[92]: Using 4 probe threads521gitlab # [ 2.030975] systemd-modules-load[92]: Inserted module 'virtio_balloon'522gitlab # [ 2.032218] systemd-modules-load[92]: Inserted module 'virtio_gpu'523gitlab # [ 2.033299] systemd-modules-load[92]: Inserted module 'dm_mod'524gitlab # [ 2.034275] systemd[1]: Starting Apply Kernel Variables...525gitlab # [ 2.035304] systemd[1]: Starting Create System Files and Directories...526gitlab # [ 2.044636] systemd-udevd[102]: Using default interface naming scheme 'v261'.527gitlab # [ 2.049595] systemd[1]: Finished Apply Kernel Variables.528gitlab # [ 2.051287] systemd[1]: Finished Create System Files and Directories.529gitlab # [ 2.059456] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 2.095180] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.096182] systemd[1]: Reached target System Initialization.532gitlab # [ 2.097042] systemd[1]: Reached target Basic System.533gitlab # [ 2.338876] uhci_hcd 0000:00:01.2: UHCI Host Controller534gitlab # [ 2.341213] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1535gitlab # [ 2.342821] uhci_hcd 0000:00:01.2: detected 2 ports536gitlab # [ 2.346246] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140537gitlab # [ 2.347656] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18538gitlab # [ 2.347966] SCSI subsystem initialized539gitlab # [ 2.348061] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1540gitlab # [ 2.350858] usb usb1: Product: UHCI Host Controller541gitlab # [ 2.351555] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd542gitlab # [ 2.352339] usb usb1: SerialNumber: 0000:00:01.2543gitlab # [ 2.354082] hub 1-0:1.0: USB hub found544gitlab # [ 2.354909] hub 1-0:1.0: 2 ports detected545gitlab # [ 2.355356] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12546gitlab # [ 2.357319] virtio_blk virtio4: 4/0/0 default/read/poll queues547gitlab # [ 2.358657] serio: i8042 KBD port at 0x60,0x64 irq 1548gitlab # [ 2.359536] serio: i8042 AUX port at 0x60,0x64 irq 12549gitlab # [ 2.368029] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)550gitlab # [ 2.375078] virtio_blk virtio5: 4/0/0 default/read/poll queues551gitlab # [ 2.378153] virtio_blk virtio5: [vdb] 11843024 512-byte logical blocks (6.06 GB/5.65 GiB)552gitlab # [ 2.395458] scsi host0: ata_piix553gitlab # [ 2.396647] scsi host1: ata_piix554gitlab # [ 2.397851] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0555gitlab # [ 2.399008] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0556gitlab # [ 2.256907] (udev-worker)[117]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.557gitlab # [ 2.259227] (udev-worker)[117]: Network interface NamePolicy= disabled on kernel command line.558gitlab # [ 2.260576] systemd[1]: Starting Virtual Console Setup...559gitlab # [ 2.262441] (udev-worker)[127]: Network interface NamePolicy= disabled on kernel command line.560gitlab # [ 2.278648] systemd[1]: Found device /dev/disk/by-label/nixos.561gitlab # [ 2.280422] systemd[1]: Reached target Initrd Root Device.562gitlab # [ 2.281418] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...563gitlab # [ 2.282712] systemd-vconsole-setup[149]: Configuration of first virtual console was skipped, ignoring remaining ones.564gitlab # [ 2.285060] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 2.287649] systemd[1]: Found device /dev/disk/by-label/nix-store.566gitlab # [ 2.308792] systemd-fsck[156]: nixos: clean, 12/65536 files, 13019/262144 blocks567gitlab # [ 2.312236] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.568gitlab # [ 2.565054] ata2: found unknown device (class 0)569gitlab # [ 2.566278] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100570gitlab # [ 2.568910] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5571gitlab # [ 2.574760] usb 1-1: new full-speed USB device number 2 using uhci_hcd572gitlab # [ 2.595131] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0573gitlab # [ 2.620951] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray574gitlab # [ 2.649309] cdrom: Uniform CD-ROM driver Revision: 3.20575gitlab # [ 2.742147] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.743397] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.744470] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.745140] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.745772] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.776486] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.790896] usbcore: registered new interface driver usbhid582gitlab # [ 2.792915] usbhid: USB HID core driver583gitlab # [ 2.803015] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.806039] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.953293] systemd[1]: Mounting /sysroot...586gitlab # [ 3.219808] EXT4-fs (vda): mounted filesystem ba6695a6-135b-4022-9d85-09d574868ac5 r/w with ordered data mode. Quota mode: none.587gitlab # [ 3.079445] systemd[1]: Mounted /sysroot.588gitlab # [ 3.081184] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 3.083229] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 3.085180] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.087092] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.088698] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.109226] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.113424] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.120152] systemd[1]: Mounted /sysroot/nix/.rw-store.596gitlab # [ 3.125342] systemd[1]: Starting rw-sysroot-nix-store.service...597gitlab # [ 3.278975] 9p: Installing v9fs 9p2000 file system support598gitlab # [ 3.133623] systemd-sysroot-fstab-check[175]: /sysroot should be mounted in the initrd, will request daemon-reload.599gitlab # [ 3.136906] systemd[1]: Reload requested from client PID 175 ('systemd-sysroot') (unit initrd-parse-etc.service)...600gitlab # [ 3.140253] systemd[1]: Reloading...601gitlab # [ 3.295557] erofs (device vdb): mounted with root inode @ nid 128.602gitlab # [ 3.202294] systemd[1]: Reloading finished in 63 ms.603gitlab # [ 3.226677] systemd-sysroot-fstab-check[175]: Requesting initrd-fs.target/start/replace...604gitlab # [ 3.229575] systemd[1]: Mounted /sysroot/nix/.ro-store.605gitlab # [ 3.231429] systemd[1]: Mounted /sysroot/run.606gitlab # [ 3.233068] systemd[1]: Mounted /sysroot/tmp/shared.607gitlab # [ 3.234768] systemd[1]: Mounted /sysroot/tmp/xchg.608gitlab # [ 3.236439] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.239075] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.241122] systemd-sysroot-fstab-check[175]: Requesting swap.target/start/replace...611gitlab # [ 3.243651] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.245645] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.247903] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.250151] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.258830] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.261456] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.953991] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 4.003928] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 4.006410] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 4.008794] systemd[1]: Starting Find NixOS closure...621gitlab # [ 4.010742] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 4.037305] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 4.042677] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 4.055644] systemd[1]: Finished Find NixOS closure.625gitlab # [ 4.057737] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 4.060260] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 4.100358] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 4.102526] systemd[1]: Stopped target Basic System.629gitlab # [ 4.104434] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 4.106470] systemd[1]: Stopped target Path Units.631gitlab # [ 4.108306] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 4.111083] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 4.113973] systemd[1]: Stopped target Slice Units.634gitlab # [ 4.115872] systemd[1]: Stopped target Socket Units.635gitlab # [ 4.117731] systemd[1]: Stopped target System Initialization.636gitlab # [ 4.120315] systemd[1]: Stopped target Swaps.637gitlab # [ 4.122927] systemd[1]: Stopped target Timer Units.638gitlab # [ 4.126388] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 4.128723] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 4.133133] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 4.136508] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.138477] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.142375] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.144524] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.146864] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.148642] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.151032] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.152503] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.154415] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.156272] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.157939] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.159436] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.160751] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.162341] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.164048] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.165322] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.167087] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.168765] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.169972] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.171333] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.172250] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.173313] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.174287] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.175203] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.176224] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.177250] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.178023] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.178843] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.179934] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.180841] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.181959] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.182973] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.183954] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.190247] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.191361] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.192541] systemd[1]: Reached target Switch Root.677gitlab # [ 4.194676] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.333088] initrd-nixos-activation-start[228]: booting system configuration /nix/store/wh90yqx9w6s2lqyv6pbi62j6c4361bxx-nixos-system-gitlab-test679gitlab # [ 4.384475] initrd-nixos-activation-start[228]: running activation script...680gitlab # [ 4.640621] initrd-nixos-activation-start[251]: setting up /etc...681gitlab # [ 4.747230] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.749413] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.750680] systemd[1]: Starting Switch Root...684gitlab # [ 4.779720] systemd[1]: Switching root.685gitlab # [ 4.982675] systemd-journald[91]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 5.104673] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.473482] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.483857] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.485765] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.487794] systemd[1]: Detected first boot.691gitlab # [ 5.490338] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.630351] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.729655] systemd[1]: Applying preset policy.694gitlab # [ 5.872140] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.990272] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.991741] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.994185] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.996481] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.997936] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.999456] systemd[1]: Created slice User and Session Slice.701gitlab # [ 6.000395] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 6.001622] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 6.002778] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 6.003556] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 6.004382] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 6.005240] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 6.005999] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 6.006848] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 6.007754] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 6.008678] systemd[1]: Reached target Path Units.711gitlab # [ 6.009448] systemd[1]: Reached target Remote File Systems.712gitlab # [ 6.010255] systemd[1]: Reached target Slice Units.713gitlab # [ 6.011009] systemd[1]: Reached target Swaps.714gitlab # [ 6.012499] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 6.014596] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 6.016008] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 6.017627] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 6.018575] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 6.021071] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 6.022534] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 6.024420] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 6.025626] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 6.026802] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 6.028130] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 6.031056] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 6.032128] systemd[1]: Listening on udev Control Socket.727gitlab # [ 6.033019] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 6.035788] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 6.037969] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 6.040129] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 6.043579] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 6.056192] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 6.057417] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 6.063693] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 6.069154] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 6.102201] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 6.106248] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 6.111833] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 6.113633] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 6.121543] systemd[1]: Starting Journal Service...741gitlab # [ 6.125195] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 6.130363] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 6.132558] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 6.133524] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.137178] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.141259] systemd[1]: Listening on Journal Log Access Socket.747gitlab # [ 6.142329] systemd[1]: Mounted Huge Pages File System.748gitlab # [ 6.142351] systemd-journald[323]: Collecting audit messages is enabled.749gitlab # [ 6.142740] systemd[1]: Mounted POSIX Message Queue File System.750gitlab # [ 6.145033] systemd[1]: Mounted Kernel Debug File System.751gitlab # [ 6.145937] systemd[1]: Mounted Kernel Trace File System.752gitlab # [ 6.147295] systemd[1]: Finished Create List of Static Device Nodes.753gitlab # [ 6.148558] systemd[1]: modprobe@configfs.service: Deactivated successfully.754gitlab # [ 6.150136] systemd[1]: Finished Load Kernel Module configfs.755gitlab # [ 6.151225] systemd[1]: Mounted FUSE Control File System.756gitlab # [ 6.154043] systemd[1]: Mounting Kernel Configuration File System...757gitlab # [ 6.159299] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...758gitlab # [ 6.161035] loop: module loaded759gitlab # [ 6.162759] EXT4-fs (vda): re-mounted ba6695a6-135b-4022-9d85-09d574868ac5.760gitlab # [ 6.020162] systemd[1]: Queued start job for default target Multi-User System.761gitlab # [ 6.167103] systemd[1]: Finished Load Kernel Modules.762gitlab # [ 6.022625] systemd[1]: Unnecessary job was removed for /dev/vdb.763gitlab # [ 6.024138] systemd[1]: systemd-journald.service: Deactivated successfully.[ 6.173439] systemd[1]: Started Journal Service.764gitlab # 765gitlab # [ 6.027924] systemd-modules-load[324]: Using 3 probe threads766gitlab # [ 6.028954] systemd-modules-load[324]: Inserted module 'tls'767gitlab # [ 6.030166] systemd-modules-load[324]: Inserted module 'loop'768gitlab # [ 6.031364] systemd[1]: Finished Remount Root and Kernel File Systems.769gitlab # [ 6.032617] systemd[1]: Mounted Kernel Configuration File System.770gitlab # [ 6.039729] systemd[1]: Listening on Disk Image Download Service Socket.771gitlab # [ 6.042183] systemd[1]: Starting Firewall...772gitlab # [ 6.043177] systemd-oomd[326]: No swap; memory pressure usage will be degraded773gitlab # [ 6.045958] systemd[1]: Starting Flush Journal to Persistent Storage...774gitlab # [ 6.047155] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore775gitlab # [ 6.052195] systemd[1]: Starting Load/Save OS Random Seed...776gitlab # [ 6.054909] systemd[1]: Starting Apply Kernel Variables...777gitlab # [ 6.055735] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os778gitlab # [ 6.058151] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.779gitlab # [ 6.060294] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.780gitlab # [ 6.210427] systemd-journald[323]: Received client request to flush runtime journal.781gitlab # [ 6.106042] systemd[1]: Starting Create Static Device Nodes in /dev...782gitlab # [ 6.108534] systemd[1]: Finished Apply Kernel Variables.783gitlab # [ 6.110351] systemd[1]: Finished Load/Save OS Random Seed.784gitlab # [ 6.112399] systemd[1]: Reached target First Boot Complete.785gitlab # [ 6.113360] systemd[1]: Finished Create Static Device Nodes in /dev.786gitlab # [ 6.114307] systemd[1]: Reached target Preparation for Local File Systems.787gitlab # [ 6.115248] systemd[1]: Starting Rule-based Manager for Device Events and Files...788gitlab # [ 6.116328] systemd[1]: Finished Flush Journal to Persistent Storage.789gitlab # [ 6.119874] systemd-udevd[356]: Using default interface naming scheme 'v261'.790gitlab # [ 6.150983] systemd[1]: Started Rule-based Manager for Device Events and Files.791gitlab # [ 6.153695] systemd[1]: Finished Coldplug All udev Devices.792gitlab # [ 6.235666] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.248892] (udev-worker)[419]: Network interface NamePolicy= disabled on kernel command line.794gitlab # [ 6.251036] (udev-worker)[426]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.795gitlab # [ 6.256093] (udev-worker)[426]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 6.258057] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.797gitlab # [ 6.268435] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.798gitlab # [ 6.278935] systemd[1]: Condition check resulted in Virtio network device being skipped.799gitlab # [ 6.280292] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore800gitlab # [ 6.281781] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67801gitlab # [ 6.283943] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore802gitlab # [ 6.285575] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os803gitlab # [ 6.286867] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.471487] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3805gitlab # [ 6.481428] ACPI: button: Power Button [PWRF]806gitlab # [ 6.523867] rtc_cmos 00:05: RTC can wake from S4807gitlab # [ 6.525793] parport_pc 00:03: reported by Plug and Play ACPI808gitlab # [ 6.527998] Floppy drive(s): fd0 is 2.88M AMI BIOS809gitlab # [ 6.528897] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]810gitlab # [ 6.535306] rtc_cmos 00:05: registered as rtc0811gitlab # [ 6.536338] rtc_cmos 00:05: setting system clock to 2026-09-06T16:30:34 UTC (1788712234)812gitlab # [ 6.537844] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs813gitlab # [ 6.538002] systemd-journald[323]: Time jumped backwards, rotating.814gitlab # [ 6.541125] FDC 0 is a S82078B815gitlab # [ 6.549598] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console816gitlab # [ 6.571173] Console: switching to colour dummy device 80x25817gitlab # [ 6.573129] [drm] Found bochs VGA, ID 0xb0c5.818gitlab # [ 6.573131] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.819gitlab # [ 6.573638] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5820gitlab # [ 6.578596] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic821gitlab # [ 6.579984] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4822gitlab # [ 6.579984] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0823gitlab # [ 6.583889] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0824gitlab # [ 6.586645] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD825gitlab # [ 6.593585] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6826gitlab # [ 6.597073] Console: switching to colour frame buffer device 160x50827gitlab # [ 6.600343] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device828gitlab # [ 6.453879] systemd[1]: Finished Firewall.829gitlab # [ 6.524382] systemd[1]: Starting Virtual Console Setup...830gitlab # [ 6.694650] mousedev: PS/2 mouse device common for all mice831gitlab # [ 6.694793] erofs (device vdb): failed to read inode meta block (nid: 128920297): -4832gitlab # [ 6.695151] ppdev: user-space parallel port driver833gitlab # [ 6.552230] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.834gitlab # [ 6.557568] systemd[1]: Stopped Virtual Console Setup.835gitlab # [ 6.564306] systemd[1]: Starting Virtual Console Setup...836gitlab # [ 6.576206] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.837gitlab # [ 6.579337] systemd[1]: Stopped Virtual Console Setup.838gitlab # [ 6.585936] systemd[1]: Starting Virtual Console Setup...839gitlab # [ 6.751480] kvm_amd: TSC scaling supported840gitlab # [ 6.752146] kvm_amd: Nested Virtualization enabled841gitlab # [ 6.752673] kvm_amd: Nested Paging enabled842gitlab # [ 6.753198] kvm_amd: LBR virtualization supported843gitlab # [ 6.753843] kvm_amd: Virtual VMLOAD VMSAVE supported844gitlab # [ 6.754415] kvm_amd: Virtual GIF supported845gitlab # [ 6.754923] kvm_amd: Virtual NMI enabled846gitlab # [ 6.789344] EDAC MC: Ver: 3.0.0847gitlab # [ 6.656194] systemd-vconsole-setup[496]: Configuration of first virtual console was skipped, ignoring remaining ones.848gitlab # [ 6.658922] systemd[1]: Finished Virtual Console Setup.849gitlab # [ 6.848827] systemd[1]: Mounting /run/wrappers...850gitlab # [ 6.877663] systemd[1]: Mounted /run/wrappers.851gitlab # [ 6.879465] systemd[1]: Reached target Local File Systems.852gitlab # [ 6.881295] systemd[1]: Listening on Boot Loader Control Service Socket.853gitlab # [ 6.884031] systemd[1]: Starting register-nix-paths.service...854gitlab # [ 6.886547] systemd[1]: Starting Create SUID/SGID Wrappers...855gitlab # [ 6.888213] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.856gitlab # [ 6.891487] systemd[1]: Starting Save Transient machine-id to Disk...857gitlab # [ 6.894922] systemd[1]: Starting Create System Files and Directories...858gitlab # [ 6.920080] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.859gitlab # [ 6.925518] systemd[1]: Finished Save Transient machine-id to Disk.860gitlab # [ 6.942153] systemd[1]: Finished Create System Files and Directories.861gitlab # [ 6.944652] systemd[1]: Starting Rebuild Journal Catalog...862gitlab # [ 6.947343] systemd[1]: Starting Record System Boot/Shutdown in UTMP...863gitlab # [ 6.964963] systemd[1]: Finished Record System Boot/Shutdown in UTMP.864gitlab # [ 6.982736] systemd[1]: Finished Rebuild Journal Catalog.865gitlab # [ 6.984301] systemd[1]: Starting Update is Completed...866gitlab # [ 6.998995] systemd[1]: Finished Update is Completed.867gitlab # [ 7.040592] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.868gitlab # [ 7.042167] systemd[1]: Finished Create SUID/SGID Wrappers.869gitlab # [ 7.299591] systemd[1]: Finished register-nix-paths.service.870gitlab # [ 7.301899] systemd[1]: Reached target System Initialization.871gitlab # [ 7.303947] systemd[1]: Started Discard unused filesystem blocks once a week.872gitlab # [ 7.306314] systemd[1]: Started logrotate.timer.873gitlab # [ 7.307965] systemd[1]: Started Daily Cleanup of Temporary Directories.874gitlab # [ 7.310257] systemd[1]: Reached target Timer Units.875gitlab # [ 7.311963] systemd[1]: Listening on D-Bus System Message Bus Socket.876gitlab # [ 7.314246] systemd[1]: Listening on Nix Daemon Socket.877gitlab # [ 7.316723] systemd[1]: Starting nixbot CI service socket...878gitlab # [ 7.318692] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.879gitlab # [ 7.321544] systemd[1]: Starting D-Bus System Message Bus...880gitlab # [ 7.324101] systemd[1]: Listening on nixbot CI service socket.881gitlab # [ 7.326184] systemd[1]: Reached target Socket Units.882gitlab # [ 7.383695] dbus-broker-launch[578]: Looking up NSS user entry for 'systemd-timesync'...883gitlab # [ 7.386808] dbus-broker-launch[578]: NSS returned no entry for 'systemd-timesync'884gitlab # [ 7.389443] dbus-broker-launch[578]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"885gitlab # [ 7.394647] systemd[1]: Started D-Bus System Message Bus.886gitlab # [ 7.415773] dbus-broker-launch[578]: Ready887gitlab # [ 7.420087] systemd[1]: Reached target Basic System.888gitlab # [ 7.423301] systemd[1]: Started backdoor.service.889gitlab # [ 7.426304] systemd[1]: Starting gitlab-config.service...890gitlab # [ 7.431141] systemd[1]: Starting Import lastlog data into lastlog2 database...891gitlab # [ 7.437225] systemd[1]: Starting Logrotate configuration check...892gitlab # [ 7.444589] systemd[1]: Starting Name Service Cache Daemon (nsncd)...893gitlab # [ 7.456431] systemd[1]: Starting Post-Boot Actions...894gitlab # [ 7.459683] systemd[1]: Started Reset console on configuration changes.895gitlab # [ 7.465211] systemd[1]: Starting resolvconf update...896gitlab # [ 7.473478] logrotate[583]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.897gitlab # [ 7.477513] logrotate[583]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf898gitlab # [ 7.480127] logrotate[583]: note: 'monthly' overrides previously specified 'weekly'899gitlab # [ 7.481915] logrotate[583]: Reading state from file: /var/lib/logrotate.status900gitlab # [ 7.484503] logrotate[583]: state file /var/lib/logrotate.status does not exist901gitlab # [ 7.486367] logrotate[583]: Allocating hash table for state file, size 64 entries902gitlab # [ 7.488347] logrotate[583]: Handling 4 logs903gitlab # connecting to host...904gitlab # [ 7.489731] logrotate[583]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed905gitlab # [ 7.492082] logrotate[583]: considering log /var/log/btmp906gitlab # [ 7.492880] logrotate[583]: Creating new state907gitlab # [ 7.493547] logrotate[583]: Now: 2026-09-06 16:30908gitlab # [ 7.494412] logrotate[583]: Last rotated at 2026-09-06 16:00909gitlab # [ 7.495164] logrotate[583]: log does not need rotating (log has already been rotated)910gitlab # [ 7.496273] logrotate[583]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed911gitlab # [ 7.498526] logrotate[583]: considering log /var/log/wtmp912gitlab # [ 7.499290] logrotate[583]: Creating new state913gitlab # [ 7.499915] logrotate[583]: Now: 2026-09-06 16:30914gitlab # [ 7.500732] logrotate[583]: Last rotated at 2026-09-06 16:00915gitlab # [ 7.501588] logrotate[583]: log does not need rotating (log has already been rotated)916gitlab # [ 7.502617] logrotate[583]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed917gitlab: Guest shell says: b'Spawning backdoor root shell...\n'918gitlab # [ 7.504318] logrotate[583]: switching euid from 0 to 165 and egid from 0 to 165 (pid 583)919gitlab # [ 7.505576] logrotate[583]: considering log /var/gitlab/state/log/*.log920gitlab: connected to guest root shell921gitlab # [ 7.506484] logrotate[583]: log /var/gitlab/state/log/*.log does not exist -- skipping922gitlab: (connecting took 8.13 seconds)923gitlab # [ 7.507519] logrotate[583]: Creating new state924gitlab: (finished: waiting for the VM to finish booting, in 8.13 seconds)925gitlab # [ 7.508161] logrotate[583]: switching euid from 165 to 0 and egid from 165 to 0 (pid 583)926gitlab # [ 7.509567] logrotate[583]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed927gitlab # [ 7.511456] logrotate[583]: switching euid from 0 to 60 and egid from 0 to 60 (pid 583)928gitlab # [ 7.512084] logrotate[583]: considering log /var/log/nginx/*.log929gitlab # [ 7.512512] logrotate[583]: log /var/log/nginx/*.log does not exist -- skipping930gitlab # [ 7.512842] logrotate[583]: Creating new state931gitlab # [ 7.513317] logrotate[583]: not running postrotate script, since no logs were rotated932gitlab # [ 7.513832] logrotate[583]: switching euid from 60 to 0 and egid from 60 to 0 (pid 583)933gitlab # [ 7.515059] nsncd[584]: Sep 06 16:30:35.604 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"934gitlab # [ 7.515811] systemd[1]: Finished Logrotate configuration check.935gitlab # [ 7.516755] systemd[1]: Started Name Service Cache Daemon (nsncd).936gitlab # [ 7.517430] systemd[1]: Finished Import lastlog data into lastlog2 database.937gitlab # [ 7.518220] systemd[1]: Reached target Host and Network Name Lookups.938gitlab # [ 7.518934] systemd[1]: Reached target User and Group Name Lookups.939gitlab # [ 7.521559] systemd[1]: Starting User Login Management...940gitlab # [ 7.522551] systemd[1]: Finished Post-Boot Actions.941gitlab # [ 7.561081] systemd-logind[615]: New seat seat0.942gitlab # [ 7.564911] systemd-logind[615]: Watching system buttons on /dev/input/event2 (Power Button)943gitlab # [ 7.565520] systemd-logind[615]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)944gitlab # [ 7.566393] systemd-logind[615]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)945gitlab # [ 7.566908] systemd[1]: Started User Login Management.946gitlab # [ 7.568460] systemd[1]: Starting linger-users.service...947gitlab # [ 7.578059] systemd[1]: Stopped target Host and Network Name Lookups.948gitlab # [ 7.578808] systemd[1]: Stopping Host and Network Name Lookups...949gitlab # [ 7.579551] systemd[1]: Stopped target User and Group Name Lookups.950gitlab # [ 7.579988] systemd[1]: Stopping User and Group Name Lookups...951gitlab # [ 7.580616] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...952gitlab # [ 7.584178] systemd[1]: nscd.service: Deactivated successfully.953gitlab # [ 7.584815] systemd[1]: Stopped Name Service Cache Daemon (nsncd).954gitlab # [ 7.585450] systemd[1]: nscd.service: Consumed 26ms CPU time over 138ms wall clock time, 7.4M memory peak, 5.5M read from disk.955gitlab # [ 7.587682] systemd[1]: linger-users.service: Deactivated successfully.956gitlab # [ 7.589564] systemd[1]: Finished linger-users.service.957gitlab # [ 7.592369] systemd[1]: Starting Name Service Cache Daemon (nsncd)...958gitlab # [ 7.604519] systemd[1]: Finished resolvconf update.959gitlab # [ 7.605341] systemd[1]: Reached target Preparation for Network.960gitlab # [ 7.608084] systemd[1]: Starting DHCP Client...961gitlab # [ 7.609589] systemd[1]: Starting Address configuration of eth1...962gitlab # [ 7.611309] systemd[1]: Starting Extra networking commands....963gitlab # [ 7.613515] nsncd[683]: Sep 06 16:30:35.723 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"964gitlab # [ 7.614715] systemd[1]: Started Name Service Cache Daemon (nsncd).965gitlab # [ 7.615756] systemd[1]: Reached target Host and Network Name Lookups.966gitlab # [ 7.616571] systemd[1]: Reached target User and Group Name Lookups.967gitlab # [ 7.640809] network-addresses-eth1-start[691]: adding address 192.168.1.1/24... done968gitlab # [ 7.646362] network-addresses-eth1-start[691]: adding address 2001:db8:1::1/64... done969gitlab # [ 7.652216] systemd[1]: Finished Address configuration of eth1.970gitlab # [ 7.680155] systemd[1]: Finished Extra networking commands..971gitlab # [ 7.681969] systemd[1]: Reached target Network.972gitlab # [ 7.684294] systemd[1]: Starting gitlab-workhorse.service...973gitlab # [ 7.687152] systemd[1]: Starting Nginx Web Server...974gitlab # [ 7.691151] systemd[1]: Starting PostgreSQL Server...975gitlab # [ 7.692936] dhcpcd[718]: dhcpcd-10.3.2 starting976gitlab # [ 7.693975] dhcpcd[769]: dev: loaded udev977gitlab # [ 7.694726] systemd[1]: Starting Redis Server - redis-gitlab...978gitlab # [ 7.847665] 8021q: 802.1Q VLAN Support v1.8979gitlab # [ 7.701646] systemd[1]: Starting Permit User Sessions...980gitlab # [ 7.849306] 8021q: adding VLAN 0 to HW filter on device eth1981gitlab # [ 7.715104] systemd[1]: Finished Permit User Sessions.982gitlab # [ 7.721237] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.983gitlab # [ 7.725979] systemd[1]: Started Getty on tty1.984gitlab # [ 7.726623] systemd[1]: Reached target Login Prompts.985gitlab # [ 7.937760] cfg80211: Loading compiled-in X.509 certificates for regulatory database986gitlab # [ 7.943079] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'987gitlab # [ 7.944031] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'988gitlab # [ 7.945275] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2989gitlab # [ 7.946171] cfg80211: failed to load regulatory.db990gitlab # [ 7.975782] 8021q: adding VLAN 0 to HW filter on device eth0991gitlab # [ 7.832779] dhcpcd[769]: eth0: waiting for carrier992gitlab # [ 7.833649] dhcpcd[769]: eth0: carrier acquired993gitlab # [ 7.836322] dhcpcd[769]: DUID 00:01:00:01:32:30:51:ab:52:54:00:12:34:56994gitlab # [ 7.837272] dhcpcd[769]: eth0: IAID 00:12:34:56995gitlab # [ 7.838182] dhcpcd[769]: eth0: adding address fe80::5054:ff:fe12:3456996gitlab # [ 7.841477] postgresql-pre-start[792]: The files belonging to this database system will be owned by user "postgres".997gitlab # [ 7.842629] postgresql-pre-start[792]: This user must also own the server process.998gitlab # [ 7.845173] nginx-pre-start[794]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok999gitlab # [ 7.845794] nginx-pre-start[794]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1000gitlab # [ 7.847034] postgresql-pre-start[792]: The database cluster will be initialized with locale "en_US.UTF-8".1001gitlab # [ 7.847444] postgresql-pre-start[792]: The default database encoding has accordingly been set to "UTF8".1002gitlab # [ 7.847957] postgresql-pre-start[792]: The default text search configuration will be set to "english".1003gitlab # [ 7.848687] postgresql-pre-start[792]: Data page checksums are enabled.1004gitlab # [ 7.849365] postgresql-pre-start[792]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1005gitlab # [ 7.849853] postgresql-pre-start[792]: creating subdirectories ... ok1006gitlab # [ 7.850663] postgresql-pre-start[792]: selecting dynamic shared memory implementation ... posix1007gitlab # [ 7.859420] systemd[1]: Started Nginx Web Server.1008gitlab # [ 7.881413] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1009gitlab # [ 7.882191] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1010gitlab # [ 7.883079] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started1011gitlab # [ 7.883647] redis-server[789]: Configuration loaded1012gitlab # [ 7.885545] redis[789]: monotonic clock: POSIX clock_gettime1013gitlab # [ 7.886203] redis[789]: Running mode=standalone, port=0.1014gitlab # [ 7.886711] redis[789]: Server initialized1015gitlab # [ 7.887214] redis[789]: Ready to accept connections unix1016gitlab # [ 7.887675] postgresql-pre-start[792]: selecting default "max_connections" ... 1001017gitlab # [ 7.889119] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1018gitlab # [ 7.889726] systemd[1]: Started Redis Server - redis-gitlab.1019gitlab # [ 7.933570] postgresql-pre-start[792]: selecting default "shared_buffers" ... 128MB1020gitlab # [ 8.205788] systemd[1]: Started gitlab-workhorse.service.1021gitlab # [ 8.330131] postgresql-pre-start[792]: selecting default time zone ... UTC1022gitlab # [ 8.331895] postgresql-pre-start[792]: creating configuration files ... ok1023gitlab # [ 8.406217] gitlab-workhorse[826]: time="2026-09-06T16:30:36Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41024gitlab # [ 8.406979] gitlab-workhorse[826]: time="2026-09-06T16:30:36Z" level=info msg="Using redis/go-redis"1025gitlab # [ 8.407618] gitlab-workhorse[826]: time="2026-09-06T16:30:36Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1026gitlab # [ 8.497918] postgresql-pre-start[792]: running bootstrap script ... ok1027gitlab # [ 8.912111] postgresql-pre-start[792]: performing post-bootstrap initialization ... ok1028gitlab # [ 9.210410] postgresql-pre-start[792]: syncing data to disk ... ok1029gitlab # [ 9.211990] postgresql-pre-start[792]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 9.212772] postgresql-pre-start[792]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 9.213910] postgresql-pre-start[792]: Success. You can now start the database server using:1032gitlab # [ 9.215144] postgresql-pre-start[792]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 9.338151] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 9.340332] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 9.341368] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 9.350458] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 9.367201] postgres[860]: [860] LOG: database system was shut down at 2026-09-06 16:30:37 GMT1038gitlab # [ 9.376798] postgres[851]: [851] LOG: database system is ready to accept connections1039gitlab # [ 9.386257] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 9.389927] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 9.611764] postgresql-setup-start[871]: CREATE DATABASE1042gitlab # [ 9.636737] postgresql-setup-start[890]: CREATE ROLE1043gitlab # [ 9.648608] postgresql-setup-start[892]: ALTER DATABASE1044gitlab # [ 9.650211] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[876]: mkdir -p /var/gitlab/state/home/.ssh: OK1045gitlab # [ 9.654531] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[876]: chmod 700 /var/gitlab/state/home/.ssh: OK1046gitlab # [ 9.668479] postgresql-setup-start[901]: CREATE ROLE1047gitlab # [ 9.671202] systemd[1]: Finished PostgreSQL Setup Scripts.1048gitlab # [ 9.672461] systemd[1]: Reached target PostgreSQL.1049gitlab # [ 9.674522] systemd[1]: Starting gitlab-postgresql.service...1050gitlab # [ 9.696081] systemd[1]: Finished gitlab-config.service.1051gitlab # [ 9.698977] systemd[1]: Started gitaly.service.1052gitlab # [ 9.701475] systemd[1]: Starting gitlab-db-config.service...1053gitlab # [ 9.737616] dhcpcd[769]: eth0: soliciting a DHCP lease1054gitlab # [ 9.894903] NET: Registered PF_PACKET protocol family1055gitlab # [ 9.758549] dhcpcd[769]: eth0: offered 10.0.2.15 from 10.0.2.21056gitlab # [ 9.763253] dhcpcd[769]: eth0: probing address 10.0.2.15/241057gitlab # [ 9.777357] gitlab-postgresql-start[919]: CREATE DATABASE1058gitlab # [ 9.808584] gitlab-postgresql-start[928]: CREATE EXTENSION1059gitlab # [ 9.891533] gitlab-postgresql-start[930]: CREATE EXTENSION1060gitlab # [ 9.895298] systemd[1]: Finished gitlab-postgresql.service.1061gitlab # [ 9.972190] dhcpcd[769]: eth0: soliciting an IPv6 router1062gitlab # [ 9.973435] dhcpcd[769]: eth0: Router Advertisement from fe80::21063gitlab # [ 9.974631] dhcpcd[769]: eth0: adding address fec0::5054:ff:fe12:3456/641064gitlab # [ 9.975960] dhcpcd[769]: eth0: adding route to fec0::/641065gitlab # [ 9.977108] dhcpcd[769]: eth0: adding default route via fe80::21066gitlab # [ 10.211712] gitaly[917]: time="2026-09-06T16:30:38.321Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9171067gitlab # [ 10.212166] gitaly[917]: time="2026-09-06T16:30:38.321Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9171068gitlab # [ 10.212865] gitaly[917]: time="2026-09-06T16:30:38.321Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9171069gitlab # [ 10.213359] gitaly[917]: time="2026-09-06T16:30:38.321Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9171070gitlab # [ 10.213840] gitaly[917]: time="2026-09-06T16:30:38.321Z" level=info msg="Starting Gitaly" pid=917 version=1071gitlab # [ 10.389119] gitaly[917]: time="2026-09-06T16:30:38.499Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=176 pid=9171072gitlab # [ 10.390093] gitaly[917]: time="2026-09-06T16:30:38.499Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9171073gitlab # [ 10.390620] gitaly[917]: time="2026-09-06T16:30:38.499Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9171074gitlab # [ 10.391091] gitaly[917]: time="2026-09-06T16:30:38.499Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-917/git-exec-672943733.d/git pid=9171075gitlab # [ 10.396525] gitaly[917]: time="2026-09-06T16:30:38.506Z" level=info msg="finished detecting git version" duration_ms=6 pid=9171076gitlab # [ 10.397573] gitaly[917]: time="2026-09-06T16:30:38.506Z" level=info msg="using Git version" pid=917 version=2.53.g1077gitlab # [ 10.398104] gitaly[917]: time="2026-09-06T16:30:38.507Z" level=info msg="clearing disk cache object folder" pid=917 storage=default1078gitlab # [ 10.398603] gitaly[917]: time="2026-09-06T16:30:38.507Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2742496453 pid=917 storage=default1079gitlab # [ 10.399090] gitaly[917]: time="2026-09-06T16:30:38.507Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2742496453 pid=917 storage=default1080gitlab # [ 10.399709] gitaly[917]: time="2026-09-06T16:30:38.507Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9171081gitlab # [ 10.400475] gitaly[917]: time="2026-09-06T16:30:38.507Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9171082gitlab # [ 10.401063] gitaly[917]: time="2026-09-06T16:30:38.507Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2742496453 clear_duration_ms=0 pid=917 storage=default1083gitlab # [ 10.457049] gitaly[917]: time="2026-09-06T16:30:38.567Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9171084gitlab # [ 10.458929] gitaly[917]: time="2026-09-06T16:30:38.568Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=917 schema=unix1085gitlab # [ 10.459853] gitaly[917]: time="2026-09-06T16:30:38.568Z" level=info msg="listening at address" address=/run/gitaly/gitaly-917/sock.d/intern pid=917 schema=unix1086gitlab # [ 10.460961] gitaly[917]: time="2026-09-06T16:30:38.568Z" level=info msg="counting repositories" pid=9171087gitlab # [ 10.462358] gitaly[917]: time="2026-09-06T16:30:38.568Z" level=info msg="starting to count repositories" pid=917 storage_path=/var/gitlab/state/repositories1088gitlab # [ 10.463578] gitaly[917]: time="2026-09-06T16:30:38.569Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=917 storage_path=/var/gitlab/state/repositories1089gitlab # [ 10.464785] gitaly[917]: time="2026-09-06T16:30:38.569Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9171090gitlab # [ 10.465797] gitaly[917]: time="2026-09-06T16:30:38.569Z" level=info msg="finished tempdir cleaner walk" pid=917 storage=default time_ms=01091gitlab # [ 10.473634] gitaly[917]: time="2026-09-06T16:30:38.583Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9171092gitlab # [ 10.474750] gitaly[917]: time="2026-09-06T16:30:38.584Z" level=info msg="Started Gitaly" duration_ms=263 pid=9171093gitlab # [ 10.475436] gitaly[917]: time="2026-09-06T16:30:38.585Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9171094gitlab # [ 10.477723] gitaly[917]: time="2026-09-06T16:30:38.587Z" level=info msg="maintenance: daily scheduled" pid=917 scheduled="2026-09-07 12:00:00 +0000 UTC"1095gitlab # [ 11.201126] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1096gitlab # [ 11.202114] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1097gitlab # [ 11.238978] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1098gitlab # [ 12.452567] gitaly[917]: time="2026-09-06T16:30:40.561Z" level=info msg="License database preloaded" duration_ms=2239 pid=9171099gitlab # [ 15.532110] dhcpcd[769]: eth0: leased 10.0.2.15 for 86400 seconds1100gitlab # [ 15.533484] dhcpcd[769]: eth0: adding route to 10.0.2.0/241101gitlab # [ 15.534721] dhcpcd[769]: eth0: adding default route via 10.0.2.21102gitlab # [ 15.611178] systemd[1]: Started DHCP Client.1103gitlab # [ 15.612176] systemd[1]: Reached target Network is Online.1104gitlab # [ 42.780751] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: W, [2026-09-06T16:31:10.890176 #922] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1105gitlab # [ 44.681459] postgres[1326]: [1326] ERROR: relation "feature_gates" does not exist at character 5231106gitlab # [ 44.682980] postgres[1326]: [1326] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1107gitlab # [ 44.684024] postgres[1326]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1108gitlab # [ 44.684366] postgres[1326]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1109gitlab # [ 44.685856] postgres[1326]: attidentity AS identity,1110gitlab # [ 44.686981] postgres[1326]: attgenerated as attgenerated1111gitlab # [ 44.688482] postgres[1326]: FROM pg_attribute a1112gitlab # [ 44.689948] postgres[1326]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1113gitlab # [ 44.691490] postgres[1326]: LEFT JOIN pg_type t ON a.atttypid = t.oid1114gitlab # [ 44.692958] postgres[1326]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1115gitlab # [ 44.694451] postgres[1326]: WHERE a.attrelid = '"feature_gates"'::regclass1116gitlab # [ 44.695947] postgres[1326]: AND a.attnum > 0 AND NOT a.attisdropped1117gitlab # [ 44.697469] postgres[1326]: ORDER BY a.attnum1118gitlab # [ 44.699047] postgres[1326]:1119gitlab # [ 48.458814] gitlab-workhorse[826]: time="2026-09-06T16:31:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1120gitlab # [ 48.729094] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1368]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1121gitlab # [ 48.730627] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1368]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1122gitlab # [ 58.467820] gitlab-workhorse[826]: time="2026-09-06T16:31:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1123gitlab # [ 63.902548] postgres[1434]: [1434] ERROR: relation "table_without_models" does not exist1124gitlab # [ 63.904217] postgres[1434]: [1434] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1125gitlab # [ 68.477675] gitlab-workhorse[826]: time="2026-09-06T16:31:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1126gitlab # [ 76.527212] systemd[1]: Finished gitlab-db-config.service.1127gitlab # [ 76.531056] systemd[1]: Started gitlab-sidekiq.service.1128gitlab # [ 76.534332] systemd[1]: Starting gitlab.service...1129gitlab # [ 76.898844] bundle[1610]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1130gitlab # [ 76.900217] bundle[1610]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1131gitlab # [ 76.901080] bundle[1610]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1132gitlab # [ 78.479437] gitlab-workhorse[826]: time="2026-09-06T16:31:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1133gitlab # [ 79.673209] sidekiq-cluster[1609]: {"severity":"INFO","time":"2026-09-06T16:31:47.765Z","message":"Starting cluster with 1 processes"}1134gitlab # [ 79.810747] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.898Z","pid":1610,"message":"* Enabling systemd notification integration"}1135gitlab # [ 79.812296] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"Puma starting in cluster mode..."}1136gitlab # [ 79.813081] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1137gitlab # [ 79.814228] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1138gitlab # [ 79.814931] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Min threads: 0"}1139gitlab # [ 79.816507] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Max threads: 4"}1140gitlab # [ 79.818078] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Environment: production"}1141gitlab # [ 79.819250] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Master PID: 1610"}1142gitlab # [ 79.820323] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Workers: 2"}1143gitlab # [ 79.821512] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1144gitlab # [ 79.822966] bundle[1610]: {"timestamp":"2026-09-06T16:31:47.920Z","pid":1610,"message":"* Preloading application"}1145gitlab # [ 88.483689] gitlab-workhorse[826]: time="2026-09-06T16:31:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 98.494163] gitlab-workhorse[826]: time="2026-09-06T16:32:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147gitlab # [ 103.178821] bundle[1610]: W, [2026-09-06T16:32:11.288219 #1610] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1148gitlab # [ 105.852179] sidekiq-cluster[1640]: 2026-09-06T16:32:13.961Z pid=1640 tid=wjk INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1640, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>436568064}1149gitlab # [ 106.027166] sidekiq-cluster[1640]: W, [2026-09-06T16:32:14.137129 #1640] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1150gitlab # [ 106.066126] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:14.176Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1151gitlab # [ 108.024510] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.133Z","message":"GitLab reliable fetch activated!"}1152gitlab # [ 108.025810] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.133Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1153gitlab # [ 108.046149] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.156Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1154gitlab # [ 108.048284] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.158Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1155gitlab # [ 108.050625] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.160Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1156gitlab # [ 108.052857] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.163Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1157gitlab # [ 108.055210] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.165Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1158gitlab # [ 108.057503] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.167Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1159gitlab # [ 108.059537] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.169Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1160gitlab # [ 108.061366] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.171Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1161gitlab # [ 108.063298] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.173Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1162gitlab # [ 108.065318] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.175Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1163gitlab # [ 108.067121] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.177Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1164gitlab # [ 108.069522] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.179Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1165gitlab # [ 108.071456] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.181Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1166gitlab # [ 108.073287] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.183Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1167gitlab # [ 108.075121] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.185Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1168gitlab # [ 108.077043] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.187Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1169gitlab # [ 108.079494] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.189Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1170gitlab # [ 108.081889] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.192Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1171gitlab # [ 108.084341] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.194Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1172gitlab # [ 108.086836] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.197Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1173gitlab # [ 108.089354] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.199Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1174gitlab # [ 108.096571] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.206Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1175gitlab # [ 108.103157] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.213Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1176gitlab # [ 108.108692] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.218Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1177gitlab # [ 108.114700] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.224Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1178gitlab # [ 108.121903] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.232Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1179gitlab # [ 108.128231] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.238Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1180gitlab # [ 108.134686] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.244Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1181gitlab # [ 108.140911] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.251Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1182gitlab # [ 108.149593] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.259Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1183gitlab # [ 108.154175] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.264Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1184gitlab # [ 108.160064] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.270Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1185gitlab # [ 108.165563] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.275Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1186gitlab # [ 108.170838] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.280Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1187gitlab # [ 108.175720] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.285Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1188gitlab # [ 108.180316] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.290Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1189gitlab # [ 108.185063] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.295Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1190gitlab # [ 108.189860] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.300Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1191gitlab # [ 108.193827] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.304Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1192gitlab # [ 108.197761] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.307Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1193gitlab # [ 108.202132] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.312Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1194gitlab # [ 108.206105] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.316Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1195gitlab # [ 108.210398] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.320Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1196gitlab # [ 108.214628] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.324Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1197gitlab # [ 108.219437] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.329Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1198gitlab # [ 108.222805] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.333Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1199gitlab # [ 108.225719] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.335Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1200gitlab # [ 108.228830] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.339Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1201gitlab # [ 108.231824] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.342Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1202gitlab # [ 108.234668] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.344Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1203gitlab # [ 108.240912] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.351Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1204gitlab # [ 108.244121] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.354Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1205gitlab # [ 108.253323] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.363Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1206gitlab # [ 108.288539] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.397Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1207gitlab # [ 108.294218] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.404Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1208gitlab # [ 108.299648] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.409Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1209gitlab # [ 108.305571] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.415Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1210gitlab # [ 108.311605] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.421Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1211gitlab # [ 108.313633] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.423Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1212gitlab # [ 108.316225] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.426Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1213gitlab # [ 108.317991] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.428Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1214gitlab # [ 108.320982] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.431Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1215gitlab # [ 108.324394] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.434Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1216gitlab # [ 108.329626] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.439Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1217gitlab # [ 108.334085] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.444Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1218gitlab # [ 108.338098] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.448Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1219gitlab # [ 108.341882] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.452Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1220gitlab # [ 108.345361] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.455Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1221gitlab # [ 108.349323] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.459Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1222gitlab # [ 108.352856] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.463Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1223gitlab # [ 108.356321] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.466Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1224gitlab # [ 108.360283] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.470Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1225gitlab # [ 108.365327] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.475Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1226gitlab # [ 108.369587] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.479Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1227gitlab # [ 108.373351] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.483Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1228gitlab # [ 108.377554] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.487Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1229gitlab # [ 108.381285] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.491Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1230gitlab # [ 108.385531] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.495Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1231gitlab # [ 108.389251] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.499Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1232gitlab # [ 108.394123] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.504Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1233gitlab # [ 108.397279] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.507Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1234gitlab # [ 108.400171] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.510Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1235gitlab # [ 108.403790] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.514Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1236gitlab # [ 108.406685] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.516Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1237gitlab # [ 108.410258] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.520Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1238gitlab # [ 108.413097] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.523Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1239gitlab # [ 108.416771] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.527Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1240gitlab # [ 108.419589] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.529Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1241gitlab # [ 108.422653] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.532Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1242gitlab # [ 108.425282] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.535Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1243gitlab # [ 108.427763] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.538Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1244gitlab # [ 108.430772] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.541Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1245gitlab # [ 108.433953] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.544Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1246gitlab # [ 108.436524] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.546Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1247gitlab # [ 108.439763] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.550Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1248gitlab # [ 108.443121] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.553Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1249gitlab # [ 108.445733] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.555Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1250gitlab # [ 108.448295] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.558Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1251gitlab # [ 108.457829] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.568Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1252gitlab # [ 108.460503] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.570Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1253gitlab # [ 108.463283] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.573Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1254gitlab # [ 108.466973] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.577Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1255gitlab # [ 108.470370] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.580Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1256gitlab # [ 108.473294] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.583Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1257gitlab # [ 108.476327] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.586Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1258gitlab # [ 108.478589] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.588Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1259gitlab # [ 108.480972] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.591Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1260gitlab # [ 108.483578] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.593Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1261gitlab # [ 108.485715] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:16.596Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1262gitlab # [ 108.502887] gitlab-workhorse[826]: time="2026-09-06T16:32:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1263gitlab # [ 118.513454] gitlab-workhorse[826]: time="2026-09-06T16:32:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1264gitlab # [ 128.515900] gitlab-workhorse[826]: time="2026-09-06T16:32:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1265gitlab # [ 130.419172] postgres[2096]: [2096] ERROR: relation "table_without_models" does not exist1266gitlab # [ 130.421364] postgres[2096]: [2096] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1267gitlab # [ 131.252993] postgres[2116]: [2116] ERROR: relation "table_without_models" does not exist1268gitlab # [ 131.254288] postgres[2116]: [2116] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1269gitlab # [ 132.808878] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:40.918Z","message":"Booted Rails 7.2.3.1 application in production environment"}1270gitlab # [ 132.816154] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:40.921Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1271gitlab # [ 132.817212] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:40.921Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1272gitlab # [ 132.818130] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:40.922Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1273gitlab # [ 132.818845] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:40.925Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1274gitlab # [ 134.013268] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.122Z","pid":1610,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1275gitlab # [ 134.014406] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.122Z","pid":1610,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1276gitlab # [ 134.015362] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.122Z","pid":1610,"message":"! #\u003cThread:0x000074e6ac5c7648 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1277gitlab # [ 134.016300] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.122Z","pid":1610,"message":"! #\u003cThread:0x000074e6ac7c44a0 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1278gitlab # [ 134.016963] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.122Z","pid":1610,"message":"Use Ctrl-C to stop"}1279gitlab # [ 134.017689] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.123Z","pid":1610,"message":"Sending status to systemd every 1.0 sec"}1280gitlab # [ 134.231615] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:42.341Z","message":"Cleaning working queues"}1281gitlab # [ 134.233896] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:42.343Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"885c78aec04494a4686f51bb","created_at":"2026-09-06T16:31:44.306Z","trace_propagation_headers":{"sentry-trace":"6caeebb4c1b74cad93524fd1e3118d44-b8ced0b44b094e28","baggage":"sentry-trace_id=6caeebb4c1b74cad93524fd1e3118d44,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"35ed6fb746e7f24f67c2443ea475aaf4","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:31:44.313Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-885c78aec04494a4686f51bb: start","job_status":"start","queue_duration_s":58.030226,"scheduling_latency_s":58.030293}1282gitlab # [ 134.361644] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:32:42.471Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"885c78aec04494a4686f51bb","created_at":"2026-09-06T16:31:44.306Z","trace_propagation_headers":{"sentry-trace":"6caeebb4c1b74cad93524fd1e3118d44-b8ced0b44b094e28","baggage":"sentry-trace_id=6caeebb4c1b74cad93524fd1e3118d44,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"35ed6fb746e7f24f67c2443ea475aaf4","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:31:44.313Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-885c78aec04494a4686f51bb: done: 0.128276 sec","job_status":"done","queue_duration_s":58.030226,"scheduling_latency_s":58.030293,"redis_calls":26,"redis_duration_s":0.003963,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002161,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.060481150000000004,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.128276,"completed_at":"2026-09-06T16:32:42.471Z","load_balancing_strategy":"primary","db_duration_s":0.03104,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1283gitlab # [ 134.706530] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.816Z","pid":1610,"message":"- Worker 0 (PID: 2173) booted in 0.65s, phase: 0"}1284gitlab # [ 134.826497] bundle[1610]: {"timestamp":"2026-09-06T16:32:42.936Z","pid":1610,"message":"- Worker 1 (PID: 2177) booted in 0.75s, phase: 0"}1285gitlab # [ 134.830989] systemd[1]: Started gitlab.service.1286gitlab # [ 134.832857] systemd[1]: Reached target Common target for all GitLab services..1287gitlab # [ 134.835310] systemd[1]: Starting setup-gitlab.service...1288gitlab # [ 134.865669] setup-gitlab-start[2217]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1289gitlab # [ 136.990112] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:32:45 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 20961290gitlab # [ 136.993420] setup-gitlab-start[2217]: + TOKEN=nixbot-test-token-of-20-chars1291gitlab # [ 136.994152] setup-gitlab-start[2217]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1292gitlab # [ 137.009902] runuser[2235]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1293gitlab # [ 137.397915] setup-gitlab-start[2236]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1294gitlab # [ 137.399552] setup-gitlab-start[2236]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1295gitlab # [ 137.401097] setup-gitlab-start[2236]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1296gitlab # [ 137.978608] setup-gitlab-start[2236]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1297gitlab # [ 137.980336] setup-gitlab-start[2236]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1298gitlab # [ 137.981374] setup-gitlab-start[2236]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1299gitlab # [ 156.639702] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:04.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"36261e9cf5498bcab960c51a","created_at":"2026-09-06T16:33:04.712Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52f35487d5814b91b28590e5f3f6aeda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:04.744Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-36261e9cf5498bcab960c51a: start","job_status":"start","queue_duration_s":0.003473,"scheduling_latency_s":0.00359}1300gitlab # [ 156.711085] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:04.821Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1301gitlab # [ 156.737835] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:04.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"36261e9cf5498bcab960c51a","created_at":"2026-09-06T16:33:04.712Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52f35487d5814b91b28590e5f3f6aeda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:04.744Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-36261e9cf5498bcab960c51a: done: 0.098568 sec","job_status":"done","queue_duration_s":0.003473,"scheduling_latency_s":0.00359,"redis_calls":10,"redis_duration_s":0.017548,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007237,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002097473,"duration_s":0.098568,"completed_at":"2026-09-06T16:33:04.846Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 156.740880] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:04.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"47b9029197ff61660c2c438b","created_at":"2026-09-06T16:33:04.785Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d33830f4f29c1f937f79fca325fc0ed4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:04.802Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-47b9029197ff61660c2c438b: start","job_status":"start","queue_duration_s":0.04719,"scheduling_latency_s":0.047229}1303gitlab # [ 156.800842] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:04.893Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"47b9029197ff61660c2c438b","created_at":"2026-09-06T16:33:04.785Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d33830f4f29c1f937f79fca325fc0ed4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:04.802Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-47b9029197ff61660c2c438b: done: 0.043163 sec","job_status":"done","queue_duration_s":0.04719,"scheduling_latency_s":0.047229,"redis_calls":12,"redis_duration_s":0.00402,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000782782,"exclusive_lock_hold_duration_s":0.013542224,"duration_s":0.043163,"completed_at":"2026-09-06T16:33:04.893Z","load_balancing_strategy":"primary","db_duration_s":0.005379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 156.857708] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:04.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"704791637bc6d805b1e447af","created_at":"2026-09-06T16:33:04.819Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc69fa315aceb965783e560d2e15cdf0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:04.964Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-704791637bc6d805b1e447af: start","job_status":"start","queue_duration_s":0.003048,"scheduling_latency_s":0.003107}1305gitlab # [ 156.957514] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"704791637bc6d805b1e447af","created_at":"2026-09-06T16:33:04.819Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc69fa315aceb965783e560d2e15cdf0","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:04.964Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-704791637bc6d805b1e447af: done: 0.100365 sec","job_status":"done","queue_duration_s":0.003048,"scheduling_latency_s":0.003107,"redis_calls":13,"redis_duration_s":0.018178,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003804,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.008707,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005667,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.016423595,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.100365,"completed_at":"2026-09-06T16:33:05.067Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01304,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1306gitlab # [ 157.052179] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.161Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a88239078513cd3c186e7b83","created_at":"2026-09-06T16:33:05.019Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cc002583e8125d97fe5028c20646441","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.031Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a88239078513cd3c186e7b83: start","job_status":"start","queue_duration_s":0.129579,"scheduling_latency_s":0.129651}1307gitlab # [ 157.130120] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.239Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a88239078513cd3c186e7b83","created_at":"2026-09-06T16:33:05.019Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cc002583e8125d97fe5028c20646441","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.031Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a88239078513cd3c186e7b83: done: 0.078697 sec","job_status":"done","queue_duration_s":0.129579,"scheduling_latency_s":0.129651,"redis_calls":10,"redis_duration_s":0.015218,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0017,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008989,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000692546,"duration_s":0.078697,"completed_at":"2026-09-06T16:33:05.239Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 157.148271] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"66626e64351b129127ca6d02","created_at":"2026-09-06T16:33:05.048Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcb43c7f17b800ad26aefa81d6cdec61","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.062Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-66626e64351b129127ca6d02: start","job_status":"start","queue_duration_s":0.195813,"scheduling_latency_s":0.195867}1309gitlab # [ 157.242020] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"66626e64351b129127ca6d02","created_at":"2026-09-06T16:33:05.048Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcb43c7f17b800ad26aefa81d6cdec61","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.062Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-66626e64351b129127ca6d02: done: 0.093688 sec","job_status":"done","queue_duration_s":0.195813,"scheduling_latency_s":0.195867,"redis_calls":12,"redis_duration_s":0.014024,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005269,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000626057,"duration_s":0.093688,"completed_at":"2026-09-06T16:33:05.351Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 157.286265] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"14f5a4b72b961d92aa43bbc0","created_at":"2026-09-06T16:33:05.160Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80657c81a0298f8caea3fb2eaa6d2c23","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.195Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-14f5a4b72b961d92aa43bbc0: start","job_status":"start","queue_duration_s":0.19999,"scheduling_latency_s":0.200033}1311gitlab # [ 157.324447] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.434Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1312gitlab # [ 157.341201] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"14f5a4b72b961d92aa43bbc0","created_at":"2026-09-06T16:33:05.160Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80657c81a0298f8caea3fb2eaa6d2c23","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.195Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-14f5a4b72b961d92aa43bbc0: done: 0.054863 sec","job_status":"done","queue_duration_s":0.19999,"scheduling_latency_s":0.200033,"redis_calls":10,"redis_duration_s":0.011338999999999998,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004016,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002541,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001971759,"duration_s":0.054863,"completed_at":"2026-09-06T16:33:05.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 157.413699] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"32eb3740ce8d3f70e237ac82","created_at":"2026-09-06T16:33:05.217Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a6add212abb63c8dfbd172d56ee5955","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:33:05.236Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-32eb3740ce8d3f70e237ac82: start","job_status":"start","queue_duration_s":0.28652,"scheduling_latency_s":0.286574}1314gitlab # [ 157.451146] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.561Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1315gitlab # [ 157.479496] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"32eb3740ce8d3f70e237ac82","created_at":"2026-09-06T16:33:05.217Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a6add212abb63c8dfbd172d56ee5955","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:33:05.236Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-32eb3740ce8d3f70e237ac82: done: 0.065893 sec","job_status":"done","queue_duration_s":0.28652,"scheduling_latency_s":0.286574,"redis_calls":10,"redis_duration_s":0.01631,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010524,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004293,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001689321,"duration_s":0.065893,"completed_at":"2026-09-06T16:33:05.589Z","load_balancing_strategy":"primary","db_duration_s":0.001029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 157.535509] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6beaf2100745d7e562c7d99d","created_at":"2026-09-06T16:33:05.252Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e653b97ff872b635fbb412c062f3c76","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.261Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6beaf2100745d7e562c7d99d: start","job_status":"start","queue_duration_s":0.383676,"scheduling_latency_s":0.383736}1317gitlab # [ 157.733973] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6beaf2100745d7e562c7d99d","created_at":"2026-09-06T16:33:05.252Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e653b97ff872b635fbb412c062f3c76","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.261Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6beaf2100745d7e562c7d99d: done: 0.197774 sec","job_status":"done","queue_duration_s":0.383676,"scheduling_latency_s":0.383736,"redis_calls":12,"redis_duration_s":0.019284,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001927,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013584,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003773,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002130717,"exclusive_lock_hold_duration_s":0.11820021900000001,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.197774,"completed_at":"2026-09-06T16:33:05.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 157.805793] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.915Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d8ed2b2eefb1c8d947795422","created_at":"2026-09-06T16:33:05.272Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"851803df72586ea5915bf9d5d857562f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.292Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d8ed2b2eefb1c8d947795422: start","job_status":"start","queue_duration_s":0.622302,"scheduling_latency_s":0.622361}1319gitlab # [ 157.886203] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:05.995Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d8ed2b2eefb1c8d947795422","created_at":"2026-09-06T16:33:05.272Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"851803df72586ea5915bf9d5d857562f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.292Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d8ed2b2eefb1c8d947795422: done: 0.080882 sec","job_status":"done","queue_duration_s":0.622302,"scheduling_latency_s":0.622361,"redis_calls":16,"redis_duration_s":0.005337,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003006,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000611,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000495315,"duration_s":0.080882,"completed_at":"2026-09-06T16:33:05.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 157.943945] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f823622a218a2cad3c74cd97","created_at":"2026-09-06T16:33:05.317Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5b7c0830455b46090dcd7be3d51abab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:33:05.332Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f823622a218a2cad3c74cd97: start","job_status":"start","queue_duration_s":0.72152,"scheduling_latency_s":0.721577}1321gitlab # [ 157.965752] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.075Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1322gitlab # [ 157.978967] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.088Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f823622a218a2cad3c74cd97","created_at":"2026-09-06T16:33:05.317Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5b7c0830455b46090dcd7be3d51abab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:33:05.332Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f823622a218a2cad3c74cd97: done: 0.035358 sec","job_status":"done","queue_duration_s":0.72152,"scheduling_latency_s":0.721577,"redis_calls":10,"redis_duration_s":0.003376,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000738,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001971,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000571861,"duration_s":0.035358,"completed_at":"2026-09-06T16:33:06.088Z","load_balancing_strategy":"primary","db_duration_s":0.000744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 158.018197] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"38c1a13ac08864cd83599693","created_at":"2026-09-06T16:33:05.347Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.399Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-38c1a13ac08864cd83599693: start","job_status":"start","queue_duration_s":0.728825,"scheduling_latency_s":0.728863}1324gitlab # [ 158.094741] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"38c1a13ac08864cd83599693","created_at":"2026-09-06T16:33:05.347Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.399Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-38c1a13ac08864cd83599693: done: 0.075521 sec","job_status":"done","queue_duration_s":0.728825,"scheduling_latency_s":0.728863,"redis_calls":19,"redis_duration_s":0.007225,"redis_read_bytes":22,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001208,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000612,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003675,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00173,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000811556,"duration_s":0.075521,"completed_at":"2026-09-06T16:33:06.203Z","load_balancing_strategy":"primary","db_duration_s":0.003567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 158.158760] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"caeaf59cf5d8a2db29e5c606","created_at":"2026-09-06T16:33:05.426Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22df38897c078d8907bfdfc863c97c6b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.442Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-caeaf59cf5d8a2db29e5c606: start","job_status":"start","queue_duration_s":0.825409,"scheduling_latency_s":0.825463}1326gitlab # [ 158.190881] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.300Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1327gitlab # [ 158.206730] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.316Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"caeaf59cf5d8a2db29e5c606","created_at":"2026-09-06T16:33:05.426Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22df38897c078d8907bfdfc863c97c6b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.442Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-caeaf59cf5d8a2db29e5c606: done: 0.04803 sec","job_status":"done","queue_duration_s":0.825409,"scheduling_latency_s":0.825463,"redis_calls":10,"redis_duration_s":0.004306,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000535,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000507886,"duration_s":0.04803,"completed_at":"2026-09-06T16:33:06.316Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 158.247650] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7549fa3adc87265364964952","created_at":"2026-09-06T16:33:05.520Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"211cb8f2384cb63bc3f67302934bf471","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.551Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7549fa3adc87265364964952: start","job_status":"start","queue_duration_s":0.805622,"scheduling_latency_s":0.805677}1329gitlab # [ 158.268476] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.378Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1330gitlab # [ 158.281163] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7549fa3adc87265364964952","created_at":"2026-09-06T16:33:05.520Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"211cb8f2384cb63bc3f67302934bf471","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.551Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7549fa3adc87265364964952: done: 0.03371 sec","job_status":"done","queue_duration_s":0.805622,"scheduling_latency_s":0.805677,"redis_calls":10,"redis_duration_s":0.0035730000000000002,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000758,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00086,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000511238,"duration_s":0.03371,"completed_at":"2026-09-06T16:33:06.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 158.283439] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"37898ace4575a86be35f4983","created_at":"2026-09-06T16:33:05.573Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80660046b1a08061ff9cf55228da2e4d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.642Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-37898ace4575a86be35f4983: start","job_status":"start","queue_duration_s":0.750819,"scheduling_latency_s":0.750851}1332gitlab # [ 158.557742] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"37898ace4575a86be35f4983","created_at":"2026-09-06T16:33:05.573Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80660046b1a08061ff9cf55228da2e4d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.642Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-37898ace4575a86be35f4983: done: 0.274169 sec","job_status":"done","queue_duration_s":0.750819,"scheduling_latency_s":0.750851,"redis_calls":565,"redis_duration_s":0.09123400000000001,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001362,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.08916,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.207264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000303111,"duration_s":0.274169,"completed_at":"2026-09-06T16:33:06.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000677,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 158.638514] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fc65d7116555bce97b06f4cd","created_at":"2026-09-06T16:33:05.672Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"419171ead063dd339a3f020aab4024a4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.691Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fc65d7116555bce97b06f4cd: start","job_status":"start","queue_duration_s":1.057127,"scheduling_latency_s":1.057166}1334gitlab # [ 158.681387] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.790Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fc65d7116555bce97b06f4cd","created_at":"2026-09-06T16:33:05.672Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"419171ead063dd339a3f020aab4024a4","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.691Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fc65d7116555bce97b06f4cd: done: 0.042803 sec","job_status":"done","queue_duration_s":1.057127,"scheduling_latency_s":1.057166,"redis_calls":12,"redis_duration_s":0.002581,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000826082,"exclusive_lock_hold_duration_s":0.004917105,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.042803,"completed_at":"2026-09-06T16:33:06.790Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 158.728109] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3b8e63f64f5bdb3a006d3a64","created_at":"2026-09-06T16:33:05.700Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.709Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3b8e63f64f5bdb3a006d3a64: start","job_status":"start","queue_duration_s":1.127886,"scheduling_latency_s":1.127929}1336gitlab # [ 158.829818] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:06.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3b8e63f64f5bdb3a006d3a64","created_at":"2026-09-06T16:33:05.700Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.709Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3b8e63f64f5bdb3a006d3a64: done: 0.101197 sec","job_status":"done","queue_duration_s":1.127886,"scheduling_latency_s":1.127929,"redis_calls":71,"redis_duration_s":0.018858,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003517,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000807,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013449,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000314845,"duration_s":0.101197,"completed_at":"2026-09-06T16:33:06.938Z","load_balancing_strategy":"primary","db_duration_s":0.003134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 158.901974] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fe871a1afafd805bd3464f28","created_at":"2026-09-06T16:33:05.740Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"989907141c842d87c5b62463119aaf9a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.749Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fe871a1afafd805bd3464f28: start","job_status":"start","queue_duration_s":1.261481,"scheduling_latency_s":1.261633}1338gitlab # [ 158.965987] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fe871a1afafd805bd3464f28","created_at":"2026-09-06T16:33:05.740Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"989907141c842d87c5b62463119aaf9a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.749Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fe871a1afafd805bd3464f28: done: 0.064154 sec","job_status":"done","queue_duration_s":1.261481,"scheduling_latency_s":1.261633,"redis_calls":10,"redis_duration_s":0.003108,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001988,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000594769,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.064154,"completed_at":"2026-09-06T16:33:07.075Z","load_balancing_strategy":"primary","db_duration_s":0.018954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 159.031948] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"24a734f9b8555fc804c67be7","created_at":"2026-09-06T16:33:05.773Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"836368456907c782a0899074c138713c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.778Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-24a734f9b8555fc804c67be7: start","job_status":"start","queue_duration_s":1.362558,"scheduling_latency_s":1.362609}1340gitlab # [ 159.083558] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"24a734f9b8555fc804c67be7","created_at":"2026-09-06T16:33:05.773Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"836368456907c782a0899074c138713c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.778Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-24a734f9b8555fc804c67be7: done: 0.05118 sec","job_status":"done","queue_duration_s":1.362558,"scheduling_latency_s":1.362609,"redis_calls":12,"redis_duration_s":0.003109,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001515,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000785854,"exclusive_lock_hold_duration_s":0.00982583,"duration_s":0.05118,"completed_at":"2026-09-06T16:33:07.192Z","load_balancing_strategy":"primary","db_duration_s":0.005206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 159.132566] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"573aeccf952d6eee87ea38f2","created_at":"2026-09-06T16:33:05.782Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f38e26a23605eec8cd6b79725422789","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.787Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-573aeccf952d6eee87ea38f2: start","job_status":"start","queue_duration_s":1.454502,"scheduling_latency_s":1.454576}1342gitlab # [ 159.341876] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"573aeccf952d6eee87ea38f2","created_at":"2026-09-06T16:33:05.782Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f38e26a23605eec8cd6b79725422789","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.787Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-573aeccf952d6eee87ea38f2: done: 0.209653 sec","job_status":"done","queue_duration_s":1.454502,"scheduling_latency_s":1.454576,"redis_calls":14,"redis_duration_s":0.005012,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001442,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001248203,"exclusive_lock_hold_duration_s":0.174728631,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.209653,"completed_at":"2026-09-06T16:33:07.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.064617,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 159.423106] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"8b9e4578a2d33ac5bdf2d7cf","created_at":"2026-09-06T16:33:05.797Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.804Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-8b9e4578a2d33ac5bdf2d7cf: start","job_status":"start","queue_duration_s":1.728006,"scheduling_latency_s":1.72804}1344gitlab # [ 159.484156] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"8b9e4578a2d33ac5bdf2d7cf","created_at":"2026-09-06T16:33:05.797Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.804Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-8b9e4578a2d33ac5bdf2d7cf: done: 0.06206 sec","job_status":"done","queue_duration_s":1.728006,"scheduling_latency_s":1.72804,"redis_calls":28,"redis_duration_s":0.006542,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000975,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000369,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.003851,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001347,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030339,"duration_s":0.06206,"completed_at":"2026-09-06T16:33:07.594Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 159.529516] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5b1a37d80f925438777c4828","created_at":"2026-09-06T16:33:05.812Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a5db2c1aaf72c18e362c74f6997b65f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.816Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5b1a37d80f925438777c4828: start","job_status":"start","queue_duration_s":1.779314,"scheduling_latency_s":1.779328}1346gitlab # [ 159.530594] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5b1a37d80f925438777c4828","created_at":"2026-09-06T16:33:05.812Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a5db2c1aaf72c18e362c74f6997b65f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.816Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5b1a37d80f925438777c4828: done: 0.0315 sec","job_status":"done","queue_duration_s":1.779314,"scheduling_latency_s":1.779328,"redis_calls":12,"redis_duration_s":0.002839,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000428826,"exclusive_lock_hold_duration_s":0.016902148,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.0315,"completed_at":"2026-09-06T16:33:07.626Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 159.576611] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b744709345aa0e0692d16e70","created_at":"2026-09-06T16:33:05.830Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21cc8a372b68a357458f3e89d0330334","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.914Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b744709345aa0e0692d16e70: start","job_status":"start","queue_duration_s":1.713917,"scheduling_latency_s":1.71393}1348gitlab # [ 159.577641] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b744709345aa0e0692d16e70","created_at":"2026-09-06T16:33:05.830Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21cc8a372b68a357458f3e89d0330334","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:05.914Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b744709345aa0e0692d16e70: done: 0.013734 sec","job_status":"done","queue_duration_s":1.713917,"scheduling_latency_s":1.71393,"redis_calls":10,"redis_duration_s":0.002143,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001237,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000306743,"duration_s":0.013734,"completed_at":"2026-09-06T16:33:07.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 159.622982] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6cb14ae869121bb93020905d","created_at":"2026-09-06T16:33:05.939Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee64cc863506782d031415de674f17ba","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:33:05.948Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6cb14ae869121bb93020905d: start","job_status":"start","queue_duration_s":1.696947,"scheduling_latency_s":1.696961}1350gitlab # [ 159.623936] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.668Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6cb14ae869121bb93020905d","created_at":"2026-09-06T16:33:05.939Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-bc31e6ddc72c4018","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee64cc863506782d031415de674f17ba","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:33:05.948Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6cb14ae869121bb93020905d: done: 0.023301 sec","job_status":"done","queue_duration_s":1.696947,"scheduling_latency_s":1.696961,"redis_calls":10,"redis_duration_s":0.0024809999999999997,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000443632,"duration_s":0.023301,"completed_at":"2026-09-06T16:33:07.668Z","load_balancing_strategy":"primary","db_duration_s":0.004515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 159.673400] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.670Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"993bfade344c3364f16851b6","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-993bfade344c3364f16851b6: start","job_status":"start","queue_duration_s":1.482436,"scheduling_latency_s":1.482448}1352gitlab # [ 159.674198] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.681Z","message":"No existing merge request to be cleaned up."}1353gitlab # [ 159.701743] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.811Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"993bfade344c3364f16851b6","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-993bfade344c3364f16851b6: done: 0.141143 sec","job_status":"done","queue_duration_s":1.482436,"scheduling_latency_s":1.482448,"redis_calls":13,"redis_duration_s":0.12100899999999999,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.118934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001591,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000278527,"duration_s":0.141143,"completed_at":"2026-09-06T16:33:07.811Z","load_balancing_strategy":"primary","db_duration_s":0.003695,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 159.739603] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.849Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c38d6fa510a07d01da98a027","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c38d6fa510a07d01da98a027: start","job_status":"start","queue_duration_s":1.661306,"scheduling_latency_s":1.661351}1355gitlab # [ 159.759474] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.869Z","message":"No existing merge request to be cleaned up."}1356gitlab # [ 159.773715] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.883Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c38d6fa510a07d01da98a027","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c38d6fa510a07d01da98a027: done: 0.034084 sec","job_status":"done","queue_duration_s":1.661306,"scheduling_latency_s":1.661351,"redis_calls":10,"redis_duration_s":0.005026,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001206,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00382,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034084,"completed_at":"2026-09-06T16:33:07.883Z","load_balancing_strategy":"primary","db_duration_s":0.003315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 159.810953] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.920Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"147df15b857e773819263fda","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-147df15b857e773819263fda: start","job_status":"start","queue_duration_s":1.732733,"scheduling_latency_s":1.732778}1358gitlab # [ 159.838955] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.948Z","message":"No existing merge request to be cleaned up."}1359gitlab # [ 159.848361] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.958Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"147df15b857e773819263fda","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-147df15b857e773819263fda: done: 0.037678 sec","job_status":"done","queue_duration_s":1.732733,"scheduling_latency_s":1.732778,"redis_calls":10,"redis_duration_s":0.002967,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037678,"completed_at":"2026-09-06T16:33:07.958Z","load_balancing_strategy":"primary","db_duration_s":0.003058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 159.884818] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:07.994Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4bbf9dde328457a17654b9d4","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4bbf9dde328457a17654b9d4: start","job_status":"start","queue_duration_s":1.806689,"scheduling_latency_s":1.806714}1361gitlab # [ 159.915534] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.025Z","message":"No existing merge request to be cleaned up."}1362gitlab # [ 159.932409] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.041Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4bbf9dde328457a17654b9d4","created_at":"2026-09-06T16:33:06.166Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-0258a995f0604aaa","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1469a8684d668700fbbae66c91cce43","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.187Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4bbf9dde328457a17654b9d4: done: 0.046956 sec","job_status":"done","queue_duration_s":1.806689,"scheduling_latency_s":1.806714,"redis_calls":10,"redis_duration_s":0.004466,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003416,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046956,"completed_at":"2026-09-06T16:33:08.041Z","load_balancing_strategy":"primary","db_duration_s":0.00339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 159.969133] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"146a096375d8206ba14f824d","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-146a096375d8206ba14f824d: start","job_status":"start","queue_duration_s":1.147799,"scheduling_latency_s":1.147842}1364gitlab # [ 160.015604] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"146a096375d8206ba14f824d","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-146a096375d8206ba14f824d: done: 0.046153 sec","job_status":"done","queue_duration_s":1.147799,"scheduling_latency_s":1.147842,"redis_calls":16,"redis_duration_s":0.004044,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002475,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000486933,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.046153,"completed_at":"2026-09-06T16:33:08.124Z","load_balancing_strategy":"primary","db_duration_s":0.000898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 160.054681] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e52521f4fa0e399a28f3eec1","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e52521f4fa0e399a28f3eec1: start","job_status":"start","queue_duration_s":1.233188,"scheduling_latency_s":1.233248}1366gitlab # [ 160.090820] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e52521f4fa0e399a28f3eec1","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e52521f4fa0e399a28f3eec1: done: 0.036209 sec","job_status":"done","queue_duration_s":1.233188,"scheduling_latency_s":1.233248,"redis_calls":13,"redis_duration_s":0.0031090000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036209,"completed_at":"2026-09-06T16:33:08.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 160.128491] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8efcaf55ea4c275d3fd7de80","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8efcaf55ea4c275d3fd7de80: start","job_status":"start","queue_duration_s":1.307199,"scheduling_latency_s":1.307245}1368gitlab # [ 160.154749] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8efcaf55ea4c275d3fd7de80","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8efcaf55ea4c275d3fd7de80: done: 0.026124 sec","job_status":"done","queue_duration_s":1.307199,"scheduling_latency_s":1.307245,"redis_calls":13,"redis_duration_s":0.0043,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026124,"completed_at":"2026-09-06T16:33:08.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 160.192750] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf81750a9601a07b418b541a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf81750a9601a07b418b541a: start","job_status":"start","queue_duration_s":1.371279,"scheduling_latency_s":1.371333}1370gitlab # [ 160.221773] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf81750a9601a07b418b541a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf81750a9601a07b418b541a: done: 0.028748 sec","job_status":"done","queue_duration_s":1.371279,"scheduling_latency_s":1.371333,"redis_calls":13,"redis_duration_s":0.004166,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002736,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028748,"completed_at":"2026-09-06T16:33:08.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 160.259834] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a9dd05b63a9a2234aea29cf","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a9dd05b63a9a2234aea29cf: start","job_status":"start","queue_duration_s":1.438504,"scheduling_latency_s":1.438548}1372gitlab # [ 160.295539] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a9dd05b63a9a2234aea29cf","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a9dd05b63a9a2234aea29cf: done: 0.035164 sec","job_status":"done","queue_duration_s":1.438504,"scheduling_latency_s":1.438548,"redis_calls":13,"redis_duration_s":0.002837,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035164,"completed_at":"2026-09-06T16:33:08.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 160.332156] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0e64d3c97a1a6056df76cb3","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0e64d3c97a1a6056df76cb3: start","job_status":"start","queue_duration_s":1.510869,"scheduling_latency_s":1.510914}1374gitlab # [ 160.367973] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0e64d3c97a1a6056df76cb3","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0e64d3c97a1a6056df76cb3: done: 0.035225 sec","job_status":"done","queue_duration_s":1.510869,"scheduling_latency_s":1.510914,"redis_calls":13,"redis_duration_s":0.003293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035225,"completed_at":"2026-09-06T16:33:08.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 160.405379] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e0385be76c7a3e4f844102e","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e0385be76c7a3e4f844102e: start","job_status":"start","queue_duration_s":1.583899,"scheduling_latency_s":1.583944}1376gitlab # [ 160.433099] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e0385be76c7a3e4f844102e","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e0385be76c7a3e4f844102e: done: 0.027791 sec","job_status":"done","queue_duration_s":1.583899,"scheduling_latency_s":1.583944,"redis_calls":13,"redis_duration_s":0.00414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001336,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002804,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027791,"completed_at":"2026-09-06T16:33:08.542Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 160.470826] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4c319b7f5f0fc2243cca498","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4c319b7f5f0fc2243cca498: start","job_status":"start","queue_duration_s":1.649562,"scheduling_latency_s":1.649606}1378gitlab # [ 160.503306] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4c319b7f5f0fc2243cca498","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4c319b7f5f0fc2243cca498: done: 0.032315 sec","job_status":"done","queue_duration_s":1.649562,"scheduling_latency_s":1.649606,"redis_calls":13,"redis_duration_s":0.007279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005678,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032315,"completed_at":"2026-09-06T16:33:08.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 160.540666] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46a24f08ec1da32436bb213b","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46a24f08ec1da32436bb213b: start","job_status":"start","queue_duration_s":1.719486,"scheduling_latency_s":1.719535}1380gitlab # [ 160.575775] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46a24f08ec1da32436bb213b","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46a24f08ec1da32436bb213b: done: 0.034923 sec","job_status":"done","queue_duration_s":1.719486,"scheduling_latency_s":1.719535,"redis_calls":13,"redis_duration_s":0.0032069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034923,"completed_at":"2026-09-06T16:33:08.685Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 160.613652] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f5bdd6ecb0471dcb02e7279","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f5bdd6ecb0471dcb02e7279: start","job_status":"start","queue_duration_s":1.791891,"scheduling_latency_s":1.791937}1382gitlab # [ 160.648595] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f5bdd6ecb0471dcb02e7279","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f5bdd6ecb0471dcb02e7279: done: 0.035166 sec","job_status":"done","queue_duration_s":1.791891,"scheduling_latency_s":1.791937,"redis_calls":13,"redis_duration_s":0.003051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001016,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035166,"completed_at":"2026-09-06T16:33:08.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 160.686524] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bda6da260fb1a5715c10b30","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bda6da260fb1a5715c10b30: start","job_status":"start","queue_duration_s":1.864918,"scheduling_latency_s":1.864969}1384gitlab # [ 160.722208] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bda6da260fb1a5715c10b30","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bda6da260fb1a5715c10b30: done: 0.035656 sec","job_status":"done","queue_duration_s":1.864918,"scheduling_latency_s":1.864969,"redis_calls":13,"redis_duration_s":0.002983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035656,"completed_at":"2026-09-06T16:33:08.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 160.758848] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bf94f4ef3f119c3742c9d81","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bf94f4ef3f119c3742c9d81: start","job_status":"start","queue_duration_s":1.937677,"scheduling_latency_s":1.937714}1386gitlab # [ 160.797281] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bf94f4ef3f119c3742c9d81","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bf94f4ef3f119c3742c9d81: done: 0.037986 sec","job_status":"done","queue_duration_s":1.937677,"scheduling_latency_s":1.937714,"redis_calls":13,"redis_duration_s":0.003554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037986,"completed_at":"2026-09-06T16:33:08.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 160.834911] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5be9d28cd62f09065863cdd","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5be9d28cd62f09065863cdd: start","job_status":"start","queue_duration_s":2.013477,"scheduling_latency_s":2.013536}1388gitlab # [ 160.863179] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:08.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5be9d28cd62f09065863cdd","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5be9d28cd62f09065863cdd: done: 0.02771 sec","job_status":"done","queue_duration_s":2.013477,"scheduling_latency_s":2.013536,"redis_calls":13,"redis_duration_s":0.004155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002825,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02771,"completed_at":"2026-09-06T16:33:08.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 160.900451] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71d7d6ac0386ae76dc9c9e19","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71d7d6ac0386ae76dc9c9e19: start","job_status":"start","queue_duration_s":2.078913,"scheduling_latency_s":2.078975}1390gitlab # [ 160.970334] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71d7d6ac0386ae76dc9c9e19","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71d7d6ac0386ae76dc9c9e19: done: 0.070163 sec","job_status":"done","queue_duration_s":2.078913,"scheduling_latency_s":2.078975,"redis_calls":13,"redis_duration_s":0.003919000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002755,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.070163,"completed_at":"2026-09-06T16:33:09.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 160.973644] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c4ae9f37f9544ede7fa4e0a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c4ae9f37f9544ede7fa4e0a: start","job_status":"start","queue_duration_s":2.152803,"scheduling_latency_s":2.152823}1392gitlab # [ 160.991585] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c4ae9f37f9544ede7fa4e0a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c4ae9f37f9544ede7fa4e0a: done: 0.017806 sec","job_status":"done","queue_duration_s":2.152803,"scheduling_latency_s":2.152823,"redis_calls":13,"redis_duration_s":0.004932000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00271,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017806,"completed_at":"2026-09-06T16:33:09.101Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 161.038784] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56972d949715ca4cbb7cf5c2","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56972d949715ca4cbb7cf5c2: start","job_status":"start","queue_duration_s":2.17368,"scheduling_latency_s":2.173695}1394gitlab # [ 161.041373] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56972d949715ca4cbb7cf5c2","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56972d949715ca4cbb7cf5c2: done: 0.010807 sec","job_status":"done","queue_duration_s":2.17368,"scheduling_latency_s":2.173695,"redis_calls":13,"redis_duration_s":0.0030269999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010807,"completed_at":"2026-09-06T16:33:09.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 161.090917] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf39e81c7eea17cc4e403c4","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf39e81c7eea17cc4e403c4: start","job_status":"start","queue_duration_s":2.186122,"scheduling_latency_s":2.186136}1396gitlab # [ 161.091848] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcf39e81c7eea17cc4e403c4","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcf39e81c7eea17cc4e403c4: done: 0.010915 sec","job_status":"done","queue_duration_s":2.186122,"scheduling_latency_s":2.186136,"redis_calls":13,"redis_duration_s":0.002763,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010915,"completed_at":"2026-09-06T16:33:09.127Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 161.124871] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86616f5c6d7f0d717d710526","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86616f5c6d7f0d717d710526: start","job_status":"start","queue_duration_s":2.198172,"scheduling_latency_s":2.198185}1398gitlab # [ 161.165767] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86616f5c6d7f0d717d710526","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86616f5c6d7f0d717d710526: done: 0.010935 sec","job_status":"done","queue_duration_s":2.198172,"scheduling_latency_s":2.198185,"redis_calls":13,"redis_duration_s":0.002749,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010935,"completed_at":"2026-09-06T16:33:09.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 161.204459] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45282cf8dffa958b6d09e104","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45282cf8dffa958b6d09e104: start","job_status":"start","queue_duration_s":2.210237,"scheduling_latency_s":2.210251}1400gitlab # [ 161.205246] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45282cf8dffa958b6d09e104","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45282cf8dffa958b6d09e104: done: 0.01766 sec","job_status":"done","queue_duration_s":2.210237,"scheduling_latency_s":2.210251,"redis_calls":13,"redis_duration_s":0.00934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007659,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01766,"completed_at":"2026-09-06T16:33:09.158Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 161.216281] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"547c63774b726a639638eb04","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-547c63774b726a639638eb04: start","job_status":"start","queue_duration_s":2.231707,"scheduling_latency_s":2.231723}1402gitlab # [ 161.257089] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"547c63774b726a639638eb04","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-547c63774b726a639638eb04: done: 0.011066 sec","job_status":"done","queue_duration_s":2.231707,"scheduling_latency_s":2.231723,"redis_calls":13,"redis_duration_s":0.002879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011066,"completed_at":"2026-09-06T16:33:09.173Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 161.262643] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a978ff0d7f6ddc7d88d55712","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a978ff0d7f6ddc7d88d55712: start","job_status":"start","queue_duration_s":2.244517,"scheduling_latency_s":2.244532}1404gitlab # [ 161.273813] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a978ff0d7f6ddc7d88d55712","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a978ff0d7f6ddc7d88d55712: done: 0.010715 sec","job_status":"done","queue_duration_s":2.244517,"scheduling_latency_s":2.244532,"redis_calls":13,"redis_duration_s":0.002788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010715,"completed_at":"2026-09-06T16:33:09.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 161.329166] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adeda614245866cb08c7cc22","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adeda614245866cb08c7cc22: start","job_status":"start","queue_duration_s":2.256399,"scheduling_latency_s":2.256436}1406gitlab # [ 161.329853] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adeda614245866cb08c7cc22","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adeda614245866cb08c7cc22: done: 0.010605 sec","job_status":"done","queue_duration_s":2.256399,"scheduling_latency_s":2.256436,"redis_calls":13,"redis_duration_s":0.0030540000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002262,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010605,"completed_at":"2026-09-06T16:33:09.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 161.392134] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09808e7a710075ae5e7465c0","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09808e7a710075ae5e7465c0: start","job_status":"start","queue_duration_s":2.268195,"scheduling_latency_s":2.268208}1408gitlab # [ 161.392880] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09808e7a710075ae5e7465c0","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09808e7a710075ae5e7465c0: done: 0.021024 sec","job_status":"done","queue_duration_s":2.268195,"scheduling_latency_s":2.268208,"redis_calls":13,"redis_duration_s":0.009978,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002393,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021024,"completed_at":"2026-09-06T16:33:09.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 161.435782] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"251d311b0cec624ac230817b","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-251d311b0cec624ac230817b: start","job_status":"start","queue_duration_s":2.292802,"scheduling_latency_s":2.292818}1410gitlab # [ 161.436652] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"251d311b0cec624ac230817b","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-251d311b0cec624ac230817b: done: 0.016179 sec","job_status":"done","queue_duration_s":2.292802,"scheduling_latency_s":2.292818,"redis_calls":13,"redis_duration_s":0.006499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001813,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004686,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016179,"completed_at":"2026-09-06T16:33:09.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 161.447726] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7b61de7d4e63ee10e5a0036","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7b61de7d4e63ee10e5a0036: start","job_status":"start","queue_duration_s":2.310541,"scheduling_latency_s":2.310555}1412gitlab # [ 161.488747] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7b61de7d4e63ee10e5a0036","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7b61de7d4e63ee10e5a0036: done: 0.009631 sec","job_status":"done","queue_duration_s":2.310541,"scheduling_latency_s":2.310555,"redis_calls":13,"redis_duration_s":0.001762,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009631,"completed_at":"2026-09-06T16:33:09.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 161.492869] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0268e0c091ac5c1ef694c20f","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0268e0c091ac5c1ef694c20f: start","job_status":"start","queue_duration_s":2.321022,"scheduling_latency_s":2.321036}1414gitlab # [ 161.506884] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0268e0c091ac5c1ef694c20f","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0268e0c091ac5c1ef694c20f: done: 0.009569 sec","job_status":"done","queue_duration_s":2.321022,"scheduling_latency_s":2.321036,"redis_calls":13,"redis_duration_s":0.001681,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009569,"completed_at":"2026-09-06T16:33:09.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 161.565043] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c3dc2a57b51fee2153ad74a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c3dc2a57b51fee2153ad74a: start","job_status":"start","queue_duration_s":2.331465,"scheduling_latency_s":2.331479}1416gitlab # [ 161.565769] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c3dc2a57b51fee2153ad74a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c3dc2a57b51fee2153ad74a: done: 0.010922 sec","job_status":"done","queue_duration_s":2.331465,"scheduling_latency_s":2.331479,"redis_calls":13,"redis_duration_s":0.002901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010922,"completed_at":"2026-09-06T16:33:09.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 161.586192] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ca3b113dd1c06f171dbded","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ca3b113dd1c06f171dbded: start","job_status":"start","queue_duration_s":2.343305,"scheduling_latency_s":2.34332}1418gitlab # [ 161.597724] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ca3b113dd1c06f171dbded","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ca3b113dd1c06f171dbded: done: 0.01063 sec","job_status":"done","queue_duration_s":2.343305,"scheduling_latency_s":2.34332,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01063,"completed_at":"2026-09-06T16:33:09.284Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 161.648794] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d9810a3a1bd0ffc655787bf","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d9810a3a1bd0ffc655787bf: start","job_status":"start","queue_duration_s":2.355359,"scheduling_latency_s":2.355373}1420gitlab # [ 161.649681] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.295Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d9810a3a1bd0ffc655787bf","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d9810a3a1bd0ffc655787bf: done: 0.009576 sec","job_status":"done","queue_duration_s":2.355359,"scheduling_latency_s":2.355373,"redis_calls":13,"redis_duration_s":0.001652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009576,"completed_at":"2026-09-06T16:33:09.295Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 161.704874] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"044c684a6b77f5707c09f656","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-044c684a6b77f5707c09f656: start","job_status":"start","queue_duration_s":2.365899,"scheduling_latency_s":2.365912}1422gitlab # [ 161.705718] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"044c684a6b77f5707c09f656","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-044c684a6b77f5707c09f656: done: 0.009529 sec","job_status":"done","queue_duration_s":2.365899,"scheduling_latency_s":2.365912,"redis_calls":13,"redis_duration_s":0.001662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009529,"completed_at":"2026-09-06T16:33:09.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 161.753847] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0584cd6de06d4b5c4ad9fafa","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0584cd6de06d4b5c4ad9fafa: start","job_status":"start","queue_duration_s":2.376293,"scheduling_latency_s":2.376307}1424gitlab # [ 161.754699] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0584cd6de06d4b5c4ad9fafa","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0584cd6de06d4b5c4ad9fafa: done: 0.014606 sec","job_status":"done","queue_duration_s":2.376293,"scheduling_latency_s":2.376307,"redis_calls":13,"redis_duration_s":0.006729000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005119,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014606,"completed_at":"2026-09-06T16:33:09.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 161.799878] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b681e524473726d0b8963fb","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b681e524473726d0b8963fb: start","job_status":"start","queue_duration_s":2.393695,"scheduling_latency_s":2.393711}1426gitlab # [ 161.800755] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b681e524473726d0b8963fb","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b681e524473726d0b8963fb: done: 0.010426 sec","job_status":"done","queue_duration_s":2.393695,"scheduling_latency_s":2.393711,"redis_calls":13,"redis_duration_s":0.002277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001743,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010426,"completed_at":"2026-09-06T16:33:09.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 161.844870] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed15abefb5706c10d8f17e5a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed15abefb5706c10d8f17e5a: start","job_status":"start","queue_duration_s":2.405341,"scheduling_latency_s":2.405355}1428gitlab # [ 161.845790] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed15abefb5706c10d8f17e5a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed15abefb5706c10d8f17e5a: done: 0.009415 sec","job_status":"done","queue_duration_s":2.405341,"scheduling_latency_s":2.405355,"redis_calls":13,"redis_duration_s":0.001601,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009415,"completed_at":"2026-09-06T16:33:09.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 161.885780] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a340a744e19718205ec45717","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a340a744e19718205ec45717: start","job_status":"start","queue_duration_s":2.416601,"scheduling_latency_s":2.416615}1430gitlab # [ 161.886491] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a340a744e19718205ec45717","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a340a744e19718205ec45717: done: 0.015686 sec","job_status":"done","queue_duration_s":2.416601,"scheduling_latency_s":2.416615,"redis_calls":13,"redis_duration_s":0.007214999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006539,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015686,"completed_at":"2026-09-06T16:33:09.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 161.924733] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2efaf01df8b01f94d743669","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2efaf01df8b01f94d743669: start","job_status":"start","queue_duration_s":2.433199,"scheduling_latency_s":2.433214}1432gitlab # [ 161.925621] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2efaf01df8b01f94d743669","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2efaf01df8b01f94d743669: done: 0.020715 sec","job_status":"done","queue_duration_s":2.433199,"scheduling_latency_s":2.433214,"redis_calls":13,"redis_duration_s":0.011916,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020715,"completed_at":"2026-09-06T16:33:09.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 161.964533] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b570795d35144cf502536fe","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b570795d35144cf502536fe: start","job_status":"start","queue_duration_s":2.456798,"scheduling_latency_s":2.456813}1434gitlab # [ 161.965474] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b570795d35144cf502536fe","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b570795d35144cf502536fe: done: 0.017543 sec","job_status":"done","queue_duration_s":2.456798,"scheduling_latency_s":2.456813,"redis_calls":13,"redis_duration_s":0.008167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006511,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017543,"completed_at":"2026-09-06T16:33:09.405Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 162.004316] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1f3f8f223799707c59c41b5","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1f3f8f223799707c59c41b5: start","job_status":"start","queue_duration_s":2.475267,"scheduling_latency_s":2.475282}1436gitlab # [ 162.009666] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1f3f8f223799707c59c41b5","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1f3f8f223799707c59c41b5: done: 0.009877 sec","job_status":"done","queue_duration_s":2.475267,"scheduling_latency_s":2.475282,"redis_calls":13,"redis_duration_s":0.001882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009877,"completed_at":"2026-09-06T16:33:09.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 162.045554] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74bb5b868fb9ec888d31743a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74bb5b868fb9ec888d31743a: start","job_status":"start","queue_duration_s":2.486095,"scheduling_latency_s":2.486108}1438gitlab # [ 162.086538] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74bb5b868fb9ec888d31743a","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74bb5b868fb9ec888d31743a: done: 0.009532 sec","job_status":"done","queue_duration_s":2.486095,"scheduling_latency_s":2.486108,"redis_calls":13,"redis_duration_s":0.0016020000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009532,"completed_at":"2026-09-06T16:33:09.426Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 162.091671] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d932777bba3ec86653d61221","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d932777bba3ec86653d61221: start","job_status":"start","queue_duration_s":2.496485,"scheduling_latency_s":2.496499}1440gitlab # [ 162.102821] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d932777bba3ec86653d61221","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d932777bba3ec86653d61221: done: 0.009444 sec","job_status":"done","queue_duration_s":2.496485,"scheduling_latency_s":2.496499,"redis_calls":13,"redis_duration_s":0.0016320000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009444,"completed_at":"2026-09-06T16:33:09.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 162.146368] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec4dd1e331ccd241bc7ea250","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec4dd1e331ccd241bc7ea250: start","job_status":"start","queue_duration_s":2.507381,"scheduling_latency_s":2.507395}1442gitlab # [ 162.148916] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec4dd1e331ccd241bc7ea250","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec4dd1e331ccd241bc7ea250: done: 0.022098 sec","job_status":"done","queue_duration_s":2.507381,"scheduling_latency_s":2.507395,"redis_calls":13,"redis_duration_s":0.01031,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022098,"completed_at":"2026-09-06T16:33:09.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 162.221140] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a98a0339b49c170a45b3f743","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a98a0339b49c170a45b3f743: start","job_status":"start","queue_duration_s":2.532775,"scheduling_latency_s":2.532791}1444gitlab # [ 162.221886] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a98a0339b49c170a45b3f743","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a98a0339b49c170a45b3f743: done: 0.009882 sec","job_status":"done","queue_duration_s":2.532775,"scheduling_latency_s":2.532791,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009882,"completed_at":"2026-09-06T16:33:09.473Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 162.229561] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65c264c6b6d47ba9fd85a500","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65c264c6b6d47ba9fd85a500: start","job_status":"start","queue_duration_s":2.54352,"scheduling_latency_s":2.543533}1446gitlab # [ 162.240821] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65c264c6b6d47ba9fd85a500","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65c264c6b6d47ba9fd85a500: done: 0.00944 sec","job_status":"done","queue_duration_s":2.54352,"scheduling_latency_s":2.543533,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00944,"completed_at":"2026-09-06T16:33:09.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 162.289512] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fd7bfd0d00f75c9cafe2518","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fd7bfd0d00f75c9cafe2518: start","job_status":"start","queue_duration_s":2.55383,"scheduling_latency_s":2.553844}1448gitlab # [ 162.290263] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fd7bfd0d00f75c9cafe2518","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fd7bfd0d00f75c9cafe2518: done: 0.0094 sec","job_status":"done","queue_duration_s":2.55383,"scheduling_latency_s":2.553844,"redis_calls":13,"redis_duration_s":0.0016150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0094,"completed_at":"2026-09-06T16:33:09.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 162.328712] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9628415c19c95cdf05b2c0ac","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9628415c19c95cdf05b2c0ac: start","job_status":"start","queue_duration_s":2.564837,"scheduling_latency_s":2.564851}1450gitlab # [ 162.368225] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9628415c19c95cdf05b2c0ac","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9628415c19c95cdf05b2c0ac: done: 0.011558 sec","job_status":"done","queue_duration_s":2.564837,"scheduling_latency_s":2.564851,"redis_calls":13,"redis_duration_s":0.0037229999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011558,"completed_at":"2026-09-06T16:33:09.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 162.369251] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08d6f4f7970201c93bcf20db","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08d6f4f7970201c93bcf20db: start","job_status":"start","queue_duration_s":2.577271,"scheduling_latency_s":2.577284}1452gitlab # [ 162.407808] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08d6f4f7970201c93bcf20db","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08d6f4f7970201c93bcf20db: done: 0.009878 sec","job_status":"done","queue_duration_s":2.577271,"scheduling_latency_s":2.577284,"redis_calls":13,"redis_duration_s":0.001869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009878,"completed_at":"2026-09-06T16:33:09.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 162.411060] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3420632189575d8e506bb1c","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3420632189575d8e506bb1c: start","job_status":"start","queue_duration_s":2.588186,"scheduling_latency_s":2.5882}1454gitlab # [ 162.456372] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3420632189575d8e506bb1c","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3420632189575d8e506bb1c: done: 0.009461 sec","job_status":"done","queue_duration_s":2.588186,"scheduling_latency_s":2.5882,"redis_calls":13,"redis_duration_s":0.001647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009461,"completed_at":"2026-09-06T16:33:09.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 162.457499] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c5cf70003df3caa9ec4c47","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c5cf70003df3caa9ec4c47: start","job_status":"start","queue_duration_s":2.598545,"scheduling_latency_s":2.598559}1456gitlab # [ 162.468682] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c5cf70003df3caa9ec4c47","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c5cf70003df3caa9ec4c47: done: 0.009367 sec","job_status":"done","queue_duration_s":2.598545,"scheduling_latency_s":2.598559,"redis_calls":13,"redis_duration_s":0.001652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009367,"completed_at":"2026-09-06T16:33:09.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 162.513691] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5167843399c3498deb3e7b8","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5167843399c3498deb3e7b8: start","job_status":"start","queue_duration_s":2.60914,"scheduling_latency_s":2.609154}1458gitlab # [ 162.514695] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5167843399c3498deb3e7b8","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5167843399c3498deb3e7b8: done: 0.016828 sec","job_status":"done","queue_duration_s":2.60914,"scheduling_latency_s":2.609154,"redis_calls":13,"redis_duration_s":0.0089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002039,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016828,"completed_at":"2026-09-06T16:33:09.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 162.571149] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8419a68178c0882f0161d243","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8419a68178c0882f0161d243: start","job_status":"start","queue_duration_s":2.627214,"scheduling_latency_s":2.627228}1460gitlab # [ 162.571821] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8419a68178c0882f0161d243","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8419a68178c0882f0161d243: done: 0.00952 sec","job_status":"done","queue_duration_s":2.627214,"scheduling_latency_s":2.627228,"redis_calls":13,"redis_duration_s":0.001725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00952,"completed_at":"2026-09-06T16:33:09.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 162.595081] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0558ea258d163be74da18f","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0558ea258d163be74da18f: start","job_status":"start","queue_duration_s":2.637611,"scheduling_latency_s":2.637625}1462gitlab # [ 162.639959] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0558ea258d163be74da18f","created_at":"2026-09-06T16:33:06.870Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-a92dbfdf04d84170","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89c9fb676a4b8527960df4e7031f97cb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:06.930Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0558ea258d163be74da18f: done: 0.008976 sec","job_status":"done","queue_duration_s":2.637611,"scheduling_latency_s":2.637625,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008976,"completed_at":"2026-09-06T16:33:09.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 162.641048] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"be78b6cab598458f227a78f7","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-be78b6cab598458f227a78f7: start","job_status":"start","queue_duration_s":1.990508,"scheduling_latency_s":1.990522}1464gitlab # [ 162.652773] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"be78b6cab598458f227a78f7","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-be78b6cab598458f227a78f7: done: 0.038055 sec","job_status":"done","queue_duration_s":1.990508,"scheduling_latency_s":1.990522,"redis_calls":16,"redis_duration_s":0.00873,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00032,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00242,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00599,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001205181,"duration_s":0.038055,"completed_at":"2026-09-06T16:33:09.616Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 162.692058] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9621e97d29fe70e81124502e","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9621e97d29fe70e81124502e: start","job_status":"start","queue_duration_s":2.032069,"scheduling_latency_s":2.032084}1466gitlab # [ 162.730938] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9621e97d29fe70e81124502e","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9621e97d29fe70e81124502e: done: 0.021205 sec","job_status":"done","queue_duration_s":2.032069,"scheduling_latency_s":2.032084,"redis_calls":13,"redis_duration_s":0.008428,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00583,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021205,"completed_at":"2026-09-06T16:33:09.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 162.734167] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"ce74bd0b8e936303a1850855","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-ce74bd0b8e936303a1850855: start","job_status":"start","queue_duration_s":2.056535,"scheduling_latency_s":2.056552}1468gitlab # [ 162.746092] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"ce74bd0b8e936303a1850855","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-ce74bd0b8e936303a1850855: done: 0.011244 sec","job_status":"done","queue_duration_s":2.056535,"scheduling_latency_s":2.056552,"redis_calls":13,"redis_duration_s":0.001902,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011244,"completed_at":"2026-09-06T16:33:09.655Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 162.811741] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f614ac9e4fa32a4b781e1907","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f614ac9e4fa32a4b781e1907: start","job_status":"start","queue_duration_s":2.068694,"scheduling_latency_s":2.068708}1470gitlab # [ 162.812662] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f614ac9e4fa32a4b781e1907","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f614ac9e4fa32a4b781e1907: done: 0.010279 sec","job_status":"done","queue_duration_s":2.068694,"scheduling_latency_s":2.068708,"redis_calls":13,"redis_duration_s":0.00166,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010279,"completed_at":"2026-09-06T16:33:09.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 162.854774] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"92fb51a9e8b150e2028779aa","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-92fb51a9e8b150e2028779aa: start","job_status":"start","queue_duration_s":2.080153,"scheduling_latency_s":2.080167}1472gitlab # [ 162.855640] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"92fb51a9e8b150e2028779aa","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-92fb51a9e8b150e2028779aa: done: 0.017781 sec","job_status":"done","queue_duration_s":2.080153,"scheduling_latency_s":2.080167,"redis_calls":13,"redis_duration_s":0.009260000000000001,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007846,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017781,"completed_at":"2026-09-06T16:33:09.685Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 162.874131] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7d9c0fa67875e1b74c83fa88","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7d9c0fa67875e1b74c83fa88: start","job_status":"start","queue_duration_s":2.100628,"scheduling_latency_s":2.100643}1474gitlab # [ 162.922580] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7d9c0fa67875e1b74c83fa88","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7d9c0fa67875e1b74c83fa88: done: 0.020306 sec","job_status":"done","queue_duration_s":2.100628,"scheduling_latency_s":2.100643,"redis_calls":13,"redis_duration_s":0.008756,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002349,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020306,"completed_at":"2026-09-06T16:33:09.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 162.923628] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d27ff9fa7dea29066387a9af","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d27ff9fa7dea29066387a9af: start","job_status":"start","queue_duration_s":2.12358,"scheduling_latency_s":2.123597}1476gitlab # [ 162.932539] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d27ff9fa7dea29066387a9af","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d27ff9fa7dea29066387a9af: done: 0.013769 sec","job_status":"done","queue_duration_s":2.12358,"scheduling_latency_s":2.123597,"redis_calls":13,"redis_duration_s":0.0038799999999999998,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013769,"completed_at":"2026-09-06T16:33:09.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 162.972163] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0c7b2781a3ee359fead1894a","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0c7b2781a3ee359fead1894a: start","job_status":"start","queue_duration_s":2.138243,"scheduling_latency_s":2.138256}1478gitlab # [ 163.010212] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0c7b2781a3ee359fead1894a","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0c7b2781a3ee359fead1894a: done: 0.118616 sec","job_status":"done","queue_duration_s":2.138243,"scheduling_latency_s":2.138256,"redis_calls":13,"redis_duration_s":0.109141,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.099972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.118616,"completed_at":"2026-09-06T16:33:09.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 163.012992] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d96918f26d85c163ed4ee279","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d96918f26d85c163ed4ee279: start","job_status":"start","queue_duration_s":2.258273,"scheduling_latency_s":2.258291}1480gitlab # [ 163.051741] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d96918f26d85c163ed4ee279","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d96918f26d85c163ed4ee279: done: 0.011908 sec","job_status":"done","queue_duration_s":2.258273,"scheduling_latency_s":2.258291,"redis_calls":13,"redis_duration_s":0.0017389999999999999,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011908,"completed_at":"2026-09-06T16:33:09.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 163.089895] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8d1447fa84c679ca6c3bd26f","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8d1447fa84c679ca6c3bd26f: start","job_status":"start","queue_duration_s":2.27102,"scheduling_latency_s":2.271034}1482gitlab # [ 163.090616] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:09.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8d1447fa84c679ca6c3bd26f","created_at":"2026-09-06T16:33:07.568Z","trace_propagation_headers":{"sentry-trace":"54f3eee6459443579c0a7ce72d5bee87-d08488504eaa4f2f","baggage":"sentry-trace_id=54f3eee6459443579c0a7ce72d5bee87,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f1c2757ba1a2eda16bd3eb0744261793","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:07.588Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8d1447fa84c679ca6c3bd26f: done: 0.016104 sec","job_status":"done","queue_duration_s":2.27102,"scheduling_latency_s":2.271034,"redis_calls":13,"redis_duration_s":0.00731,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006781,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016104,"completed_at":"2026-09-06T16:33:09.875Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000919,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 163.779828] setup-gitlab-start[2236]: W, [2026-09-06T16:33:11.889134 #2236] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1484gitlab # [ 185.179352] postgres[2444]: [2444] ERROR: relation "table_without_models" does not exist1485gitlab # [ 185.180578] postgres[2444]: [2444] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1486gitlab # [ 188.058544] runuser[2235]: pam_unix(runuser:session): session closed for user gitlab1487gitlab # [ 188.060292] setup-gitlab-start[2217]: + systemctl restart gitlab-sidekiq.service1488gitlab # [ 188.080981] systemd[1]: Stopping gitlab-sidekiq.service...1489gitlab # [ 188.082136] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:36.191Z","message":"Shutting down"}1490gitlab # [ 188.083522] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:36.191Z","message":"Scheduler exiting..."}1491gitlab # [ 188.084924] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:36.192Z","message":"Terminating quiet threads for default capsule"}1492gitlab # [ 188.086575] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:36.192Z","message":"Scheduler exiting..."}1493gitlab # [ 188.584121] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:36.693Z","message":"Pausing to allow jobs to finish..."}1494gitlab # [ 192.088910] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:40.198Z","message":"Bye!"}1495gitlab # [ 192.117359] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-09-06T16:33:40.227Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1640,"worker_id":"sidekiq_0","memwd_rss_bytes":923959296,"retry":0}1496gitlab # [ 192.721150] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1497gitlab # [ 192.722419] systemd[1]: Stopped gitlab-sidekiq.service.1498gitlab # [ 192.723366] systemd[1]: gitlab-sidekiq.service: Consumed 57.655s CPU time over 1min 56.189s wall clock time, 956.6M memory peak, 5M read from disk.1499gitlab # [ 192.726866] systemd[1]: Started gitlab-sidekiq.service.1500gitlab # [ 192.729979] setup-gitlab-start[2217]: + mkdir -p /var/lib/secrets1501gitlab # [ 192.735380] setup-gitlab-start[2217]: + echo nixbot-test-token-of-20-chars1502gitlab # [ 192.736621] setup-gitlab-start[2217]: + chmod 644 /var/lib/secrets/gitlab-token1503gitlab # [ 192.739628] setup-gitlab-start[2217]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1504gitlab # [ 192.990135] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:41 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 2181505gitlab # [ 193.532278] gitaly[917]: time="2026-09-06T16:33:41.641Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.641" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1506gitlab # [ 193.533755] gitaly[917]: time="2026-09-06T16:33:41.641Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.641" grpc.time_ms=0.445 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1507gitlab # [ 193.659689] gitaly[917]: time="2026-09-06T16:33:41.769Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.129" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.769" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1508gitlab # [ 193.660971] gitaly[917]: time="2026-09-06T16:33:41.770Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.129" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.769" grpc.time_ms=0.266 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1509gitlab # [ 193.720595] gitaly[917]: time="2026-09-06T16:33:41.830Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N diskcache=721a7cb9-0648-4a24-b9fc-58365edc71f7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:33:51.137" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.778" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1510gitlab # [ 193.730296] gitaly[917]: time="2026-09-06T16:33:41.831Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:33:51.137" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.778" grpc.time_ms=52.423 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1511gitlab # [ 193.761638] gitaly[917]: time="2026-09-06T16:33:41.871Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.871" pid=917 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1512gitlab # [ 193.762916] gitaly[917]: time="2026-09-06T16:33:41.872Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.871" grpc.time_ms=0.881 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1513gitlab # [ 193.798544] gitaly[917]: time="2026-09-06T16:33:41.908Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:33:51.257" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.897" grpc.time_ms=10.543 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1514gitlab # [ 193.866157] gitaly[917]: time="2026-09-06T16:33:41.976Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.976" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1515gitlab # [ 193.866857] gitaly[917]: time="2026-09-06T16:33:41.976Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:51.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.976" grpc.time_ms=0.3 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1516gitlab # [ 193.928444] gitaly[917]: time="2026-09-06T16:33:42.038Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N diskcache=98656284-acb1-463b-b8bf-0972d58358e6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:33:51.027" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.998" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1517gitlab # [ 193.929852] gitaly[917]: time="2026-09-06T16:33:42.038Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1582 command.oublock=32 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:33:51.027" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:41.998" grpc.time_ms=40.482 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1518gitlab # [ 193.960141] gitaly[917]: time="2026-09-06T16:33:42.070Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:52.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:42.069" pid=917 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1519gitlab # [ 193.961340] gitaly[917]: time="2026-09-06T16:33:42.070Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:52.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:42.069" grpc.time_ms=0.892 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1520gitlab # [ 193.995899] gitaly[917]: time="2026-09-06T16:33:42.106Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:33:52.034" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:42.094" grpc.time_ms=10.935 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1521gitlab # [ 194.048560] gitaly[917]: time="2026-09-06T16:33:42.158Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:33:52.097" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:42.158" grpc.time_ms=0.236 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1522gitlab # [ 194.061856] gitaly[917]: time="2026-09-06T16:33:42.171Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:33:52.110" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:42.171" grpc.time_ms=0.603 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1523gitlab # [ 194.451672] gitaly[917]: time="2026-09-06T16:33:42.561Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS31R09QBJC2M2X488RC4N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:33:52.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:42.555" grpc.time_ms=5.813 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1524gitlab # [ 194.818517] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:42 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 18071525gitlab # [ 194.820606] setup-gitlab-start[2499]: {"id":1,"message":null,"error":null}1526gitlab # [ 195.891686] sidekiq-cluster[2487]: {"severity":"INFO","time":"2026-09-06T16:33:43.984Z","message":"Starting cluster with 1 processes"}1527gitlab # [ 199.958195] gitaly[917]: time="2026-09-06T16:33:48.066Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38DMGZY15FZ3Q0S98TQJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:33:58.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:48.060" grpc.time_ms=6.246 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1528gitlab # [ 200.029469] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:48 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1821529gitlab # [ 200.033473] setup-gitlab-start[2217]: + mkdir -p /tmp/test-flake1530gitlab # [ 200.035349] setup-gitlab-start[2217]: + cd /tmp/test-flake1531gitlab # [ 200.036276] setup-gitlab-start[2217]: + git init -b master1532gitlab # [ 200.057782] setup-gitlab-start[2572]: Initialized empty Git repository in /tmp/test-flake/.git/1533gitlab # [ 200.059026] setup-gitlab-start[2217]: + git config user.name test1534gitlab # [ 200.060182] setup-gitlab-start[2217]: + git config user.email test@example.com1535gitlab # [ 200.062290] setup-gitlab-start[2217]: + cat1536gitlab # [ 200.064357] setup-gitlab-start[2217]: + git add flake.nix1537gitlab # [ 200.066674] setup-gitlab-start[2217]: + git commit -m 'initial commit'1538gitlab # [ 200.082072] setup-gitlab-start[2577]: [master (root-commit) e485f2c] initial commit1539gitlab # [ 200.083166] setup-gitlab-start[2577]: 1 file changed, 10 insertions(+)1540gitlab # [ 200.084054] setup-gitlab-start[2577]: create mode 100644 flake.nix1541gitlab # [ 200.085113] setup-gitlab-start[2217]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1542gitlab # [ 200.088411] setup-gitlab-start[2217]: + git push -u origin master1543gitlab # [ 200.139204] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:48 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 311544gitlab # [ 200.228869] gitaly[917]: time="2026-09-06T16:33:48.338Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38PT205ZY91WFTHR9EF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:48.331" grpc.time_ms=7.356 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1545gitlab # [ 200.241299] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:48 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 891546gitlab # [ 200.455419] gitaly[917]: time="2026-09-06T16:33:48.565Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:33:58.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:48.555" grpc.time_ms=10.32 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 200.492280] gitaly[917]: time="2026-09-06T16:33:48.602Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:34:18.095" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:33:48.595" grpc.time_ms=6.43 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1548gitlab # [ 200.518031] gitaly[917]: time="2026-09-06T16:33:48.628Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 duration_ms=127ns method=POST pid=917 status=200 url="http://unix/api/v4/internal/allowed"1549gitlab # [ 200.683930] gitaly[917]: time="2026-09-06T16:33:48.794Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 duration_ms=162ns method=POST pid=917 status=200 url="http://unix/api/v4/internal/pre_receive"1550gitlab # [ 200.684858] gitaly[917]: time="2026-09-06T16:33:48.794Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5308 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:48.499" grpc.time_ms=294.88 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1551gitlab # [ 200.748085] gitaly[917]: time="2026-09-06T16:33:48.858Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:48.857" grpc.time_ms=0.621 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1552gitlab # [ 200.800180] gitaly[917]: time="2026-09-06T16:33:48.910Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4777 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:48.909" grpc.time_ms=0.745 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1553gitlab # [ 200.859107] gitaly[917]: time="2026-09-06T16:33:48.969Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:48.968" grpc.time_ms=0.447 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1554gitlab # [ 200.972921] gitaly[917]: time="2026-09-06T16:33:49.082Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:33:59.011" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:49.071" grpc.time_ms=9.195 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1555gitlab # [ 201.032371] gitaly[917]: time="2026-09-06T16:33:49.142Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:33:59.055" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:33:49.116" grpc.time_ms=25.715 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1556gitlab # [ 201.064923] gitaly[917]: time="2026-09-06T16:33:49.175Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 duration_ms=159ns method=POST pid=917 status=200 url="http://unix/api/v4/internal/post_receive"1557gitlab # [ 201.071107] gitaly[917]: time="2026-09-06T16:33:49.181Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4802 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:49.014" grpc.time_ms=166.192 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1558gitlab # [ 201.096380] gitaly[917]: time="2026-09-06T16:33:49.206Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 diskcache=b5bc0d41-c369-430d-82c9-14e3fa737a79 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:48.438" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1559gitlab # [ 201.097940] gitlab-workhorse[826]: time="2026-09-06T16:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521560gitlab # [ 201.098593] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:49 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8491561gitlab # [ 201.109153] gitaly[917]: time="2026-09-06T16:33:49.219Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=210 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13764 command.oublock=32 command.real_time_ms=757 command.spawn_token_fork_ms=0 command.system_time_ms=148 command.user_time_ms=62 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:48.438" grpc.time_ms=768.525 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1562gitlab # [ 201.121824] setup-gitlab-start[2581]: To http://gitlab/root/test-flake.git1563gitlab # [ 201.122357] setup-gitlab-start[2581]: * [new branch] master -> master1564gitlab # [ 201.128142] setup-gitlab-start[2581]: branch 'master' set up to track 'origin/master'.1565gitlab # [ 201.132076] systemd[1]: Finished setup-gitlab.service.1566gitlab # [ 201.136602] systemd[1]: Starting nixbot CI service...1567gitlab: (finished: waiting for unit setup-gitlab.service, in 202.13 seconds)1568gitlab: waiting for unit nixbot.service1569gitlab # [ 202.344715] nixbot[2660]: {"ts": "2026-09-06T16:33:50.454Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1570gitlab # [ 202.366853] nixbot[2660]: {"ts": "2026-09-06T16:33:50.477Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1571gitlab # [ 202.398045] nixbot[2660]: {"ts": "2026-09-06T16:33:50.508Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1572gitlab # [ 202.405344] nixbot[2660]: {"ts": "2026-09-06T16:33:50.515Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1573gitlab # [ 202.410856] nixbot[2660]: {"ts": "2026-09-06T16:33:50.521Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1574gitlab # [ 202.420806] nixbot[2660]: {"ts": "2026-09-06T16:33:50.531Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1575gitlab # [ 202.428936] nixbot[2660]: {"ts": "2026-09-06T16:33:50.539Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1576gitlab # [ 202.435121] nixbot[2660]: {"ts": "2026-09-06T16:33:50.545Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1577gitlab # [ 202.444934] nixbot[2660]: {"ts": "2026-09-06T16:33:50.554Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1578gitlab # [ 202.451936] nixbot[2660]: {"ts": "2026-09-06T16:33:50.562Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1579gitlab # [ 202.462829] nixbot[2660]: {"ts": "2026-09-06T16:33:50.572Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1580gitlab # [ 202.471903] nixbot[2660]: {"ts": "2026-09-06T16:33:50.582Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1581gitlab # [ 202.480486] nixbot[2660]: {"ts": "2026-09-06T16:33:50.590Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1582gitlab # [ 202.486687] nixbot[2660]: {"ts": "2026-09-06T16:33:50.596Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1583gitlab # [ 202.495952] nixbot[2660]: {"ts": "2026-09-06T16:33:50.606Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1584gitlab # [ 202.502243] nixbot[2660]: {"ts": "2026-09-06T16:33:50.612Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1585gitlab # [ 202.508142] nixbot[2660]: {"ts": "2026-09-06T16:33:50.618Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1586gitlab # [ 202.514287] nixbot[2660]: {"ts": "2026-09-06T16:33:50.624Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1587gitlab # [ 202.520656] nixbot[2660]: {"ts": "2026-09-06T16:33:50.630Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1588gitlab # [ 202.524941] nixbot[2660]: {"ts": "2026-09-06T16:33:50.635Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1589gitlab # [ 202.529055] nixbot[2660]: {"ts": "2026-09-06T16:33:50.639Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1590gitlab # [ 202.535189] nixbot[2660]: {"ts": "2026-09-06T16:33:50.645Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1591gitlab # [ 202.537483] nixbot[2660]: {"ts": "2026-09-06T16:33:50.647Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1592gitlab # [ 202.539862] nixbot[2660]: {"ts": "2026-09-06T16:33:50.650Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1593gitlab # [ 202.541934] nixbot[2660]: {"ts": "2026-09-06T16:33:50.652Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1594gitlab # [ 202.544431] nixbot[2660]: {"ts": "2026-09-06T16:33:50.654Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1595gitlab # [ 202.548805] nixbot[2660]: {"ts": "2026-09-06T16:33:50.659Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1596gitlab # [ 202.551233] nixbot[2660]: {"ts": "2026-09-06T16:33:50.660Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1597gitlab # [ 202.554534] nixbot[2660]: {"ts": "2026-09-06T16:33:50.664Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1598gitlab # [ 202.563757] nixbot[2660]: {"ts": "2026-09-06T16:33:50.673Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1599gitlab # [ 202.573644] nixbot[2660]: {"ts": "2026-09-06T16:33:50.683Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1600gitlab # [ 202.577690] nixbot[2660]: {"ts": "2026-09-06T16:33:50.687Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1601gitlab # [ 202.580316] nixbot[2660]: {"ts": "2026-09-06T16:33:50.690Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "public_badges"}1602gitlab # [ 202.880399] nixbot[2660]: INFO: Started server process [2660]1603gitlab # [ 202.882419] nixbot[2660]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1604gitlab # [ 202.900529] nixbot[2660]: INFO: ::ffff:127.0.0.1:35326 - "GET /health HTTP/1.1" 200 OK1605gitlab # [ 202.906345] systemd[1]: Started nixbot CI service.1606gitlab # [ 202.910758] systemd[1]: Reached target Multi-User System.1607gitlab # [ 202.913583] systemd[1]: Startup finished in 1.243s (kernel) + 3.680s (initrd) + 3min 17.988s (userspace) = 3min 22.912s.1608gitlab # [ 203.162070] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:51 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2591609gitlab # [ 203.163781] nixbot[2660]: {"ts": "2026-09-06T16:33:51.273Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1610gitlab # [ 203.180810] nixbot[2660]: {"ts": "2026-09-06T16:33:51.290Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1611gitlab # [ 203.289689] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 941612gitlab # [ 203.291631] nixbot[2660]: {"ts": "2026-09-06T16:33:51.401Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1613gitlab # [ 203.294117] nixbot[2660]: {"ts": "2026-09-06T16:33:51.404Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1614gitlab # [ 203.473173] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1741615gitlab # [ 203.478826] nixbot[2660]: {"ts": "2026-09-06T16:33:51.584Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1616gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1617gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1618gitlab # [ 203.630264] nixbot[2660]: INFO: ::ffff:127.0.0.1:58056 - "GET /health HTTP/1.1" 200 OK1619gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds)1620subtest: project discovered and webhook registered1621gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1622gitlab # [ 203.730644] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 761623gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1624(finished: subtest: project discovered and webhook registered, in 0.11 seconds)1625subtest: push delivers the webhook1626gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1627gitlab # [ 204.079722] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 5971628gitlab # [ 204.086309] nixbot[2660]: {"ts": "2026-09-06T16:33:52.196Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1629gitlab # [ 204.141943] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 371630gitlab # [ 204.143270] nixbot[2660]: {"ts": "2026-09-06T16:33:52.253Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1631gitlab # [ 204.189977] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 401632gitlab # [ 204.194914] nixbot[2660]: {"ts": "2026-09-06T16:33:52.300Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1633gitlab # [ 204.251834] gitaly[917]: time="2026-09-06T16:33:52.361Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3CNJADD5E32AGYDPVJT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-06T16:34:22.046" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:33:52.346" grpc.time_ms=14.724 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1634gitlab # [ 204.294411] gitaly[917]: time="2026-09-06T16:33:52.404Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3CNJADD5E32AGYDPVJT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:34:22.103" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:33:52.403" grpc.time_ms=0.81 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1635gitlab # [ 204.329051] gitaly[917]: time="2026-09-06T16:33:52.439Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3CNJADD5E32AGYDPVJT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-06T16:34:02.006" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:33:52.428" grpc.time_ms=10.258 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1636gitlab # [ 204.380625] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1831637gitlab # [ 204.381698] nixbot[2660]: {"ts": "2026-09-06T16:33:52.491Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1638gitlab # [ 204.451074] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 641639gitlab # [ 204.452268] nixbot[2660]: {"ts": "2026-09-06T16:33:52.561Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1640gitlab # [ 204.456389] nixbot[2660]: {"ts": "2026-09-06T16:33:52.565Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1641gitlab # [ 204.540051] gitaly[917]: time="2026-09-06T16:33:52.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3CYZDN3H1GB399D81Y52 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.643" grpc.time_ms=6.729 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1642gitlab # [ 204.552412] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 551643gitlab # [ 204.648760] gitaly[917]: time="2026-09-06T16:33:52.758Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3D0SRWTJTSH7RQ1QPDEF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.751" pid=917 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1644gitlab # [ 204.649559] gitaly[917]: time="2026-09-06T16:33:52.759Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3D0SRWTJTSH7RQ1QPDEF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.751" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1645gitlab # [ 204.650205] gitaly[917]: time="2026-09-06T16:33:52.759Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3D0SRWTJTSH7RQ1QPDEF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.751" grpc.time_ms=7.262 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1646gitlab # [ 204.682338] gitlab-workhorse[826]: time="2026-09-06T16:33:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131647gitlab # [ 204.683021] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1271648gitlab # [ 204.743333] gitaly[917]: time="2026-09-06T16:33:52.853Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3D4Z2DZNJKCMB89GZDQ3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.837" pid=917 remote_ip="2001:db8:1::1" request_sha=366b04936e421802d40c80527815df6e4d70f2c0 response_bytes=223 span.kind=server system=grpc1649gitlab # [ 204.743803] gitaly[917]: time="2026-09-06T16:33:52.853Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3D4Z2DZNJKCMB89GZDQ3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.837" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1650gitlab # [ 204.764724] gitaly[917]: time="2026-09-06T16:33:52.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3D4Z2DZNJKCMB89GZDQ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.837" grpc.time_ms=36.926 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1651gitlab # [ 204.780074] gitlab-workhorse[826]: time="2026-09-06T16:33:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231652gitlab # [ 204.780729] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 891653gitlab # [ 204.873415] gitaly[917]: time="2026-09-06T16:33:52.983Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3D99GTWBYDEN4EDAB4ZA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:52.976" grpc.time_ms=6.79 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1654gitlab # [ 204.875377] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 481655gitlab # [ 204.967822] gitaly[917]: time="2026-09-06T16:33:53.077Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DB6TJ9PK53YFR74TRHF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.071" pid=917 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1656gitlab # [ 204.968897] gitaly[917]: time="2026-09-06T16:33:53.078Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DB6TJ9PK53YFR74TRHF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.071" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1657gitlab # [ 204.969764] gitaly[917]: time="2026-09-06T16:33:53.078Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DB6TJ9PK53YFR74TRHF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.071" grpc.time_ms=6.627 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1658gitlab # [ 204.970696] gitlab-workhorse[826]: time="2026-09-06T16:33:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671659gitlab # [ 204.971249] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 801660gitlab # [ 205.068613] gitaly[917]: time="2026-09-06T16:33:53.178Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DFFA3QEE07J2AS53RD4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.170" grpc.time_ms=8.268 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1661gitlab # [ 205.070505] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:53 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 441662gitlab # [ 205.136444] gitaly[917]: time="2026-09-06T16:33:53.246Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DH9Y8B8XA5H8C0AJC4C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.232" pid=917 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1663gitlab # [ 205.137434] gitaly[917]: time="2026-09-06T16:33:53.247Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DH9Y8B8XA5H8C0AJC4C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.232" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1664gitlab # [ 205.138135] gitaly[917]: time="2026-09-06T16:33:53.247Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DH9Y8B8XA5H8C0AJC4C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:33:53.232" grpc.time_ms=14.756 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1665gitlab # [ 205.139469] gitlab-workhorse[826]: time="2026-09-06T16:33:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091666gitlab # [ 205.140646] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:53 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 561667gitlab # [ 205.281753] gitaly[917]: time="2026-09-06T16:33:53.389Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:34:23.088" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:33:53.388" grpc.time_ms=0.781 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1668gitlab # [ 205.334194] gitaly[917]: time="2026-09-06T16:33:53.442Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:34:23.133" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:33:53.434" grpc.time_ms=7.939 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1669gitlab # [ 205.370315] gitaly[917]: time="2026-09-06T16:33:53.480Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:03.071" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:33:53.472" grpc.time_ms=7.943 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670gitlab # [ 205.403398] gitaly[917]: time="2026-09-06T16:33:53.513Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:03.103" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:33:53.503" grpc.time_ms=9.919 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671gitlab # [ 205.497147] gitaly[917]: time="2026-09-06T16:33:53.607Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:34:03.206" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:53.606" grpc.time_ms=0.277 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1672gitlab # [ 205.523337] gitaly[917]: time="2026-09-06T16:33:53.633Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:34:03.222" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:33:53.623" grpc.time_ms=9.65 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1673gitlab # [ 205.585805] gitaly[917]: time="2026-09-06T16:33:53.695Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4147 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:53.695" grpc.time_ms=0.397 pid=917 span.kind=server system=grpc1674gitlab # [ 205.623795] gitaly[917]: time="2026-09-06T16:33:53.733Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:33:53.732" grpc.time_ms=0.282 pid=917 span.kind=server system=grpc1675gitlab # [ 205.643358] gitaly[917]: time="2026-09-06T16:33:53.753Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F diskcache=4e28e00c-706f-42ab-b576-641a39e96168 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:34:03.247" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:53.647" pid=917 ref=refs/keep-around/e485f2cbba70e6f175cf8098620ece9de959d0ad remote_ip="2001:db8:1::1" revision=e485f2cbba70e6f175cf8098620ece9de959d0ad span.kind=server system=grpc username=root1676gitlab # [ 205.644658] gitaly[917]: time="2026-09-06T16:33:53.753Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=88 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5384 command.oublock=8 command.real_time_ms=95 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:34:03.247" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:33:53.647" grpc.time_ms=105.899 pid=917 ref=refs/keep-around/e485f2cbba70e6f175cf8098620ece9de959d0ad remote_ip="2001:db8:1::1" revision=e485f2cbba70e6f175cf8098620ece9de959d0ad span.kind=server system=grpc username=root1677gitlab # [ 205.650457] gitaly[917]: time="2026-09-06T16:33:53.760Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:34:23.459" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:33:53.760" grpc.time_ms=0.483 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678gitlab # [ 205.792261] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:33:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5731679gitlab # [ 205.793696] nixbot[2660]: {"ts": "2026-09-06T16:33:53.903Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1680gitlab # [ 205.804925] nixbot[2660]: {"ts": "2026-09-06T16:33:53.914Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-e485f2cbba70-0"}1681gitlab # [ 205.965440] nixbot[2660]: {"ts": "2026-09-06T16:33:54.075Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e485f2cbba70-0", "argv0": "bwrap"}1682gitlab # [ 206.042948] systemd[1]: Started Nix Daemon.1683gitlab # [ 206.086292] nix-daemon[2834]: accepted connection from pid 2829, user nixbot1684gitlab # [ 210.718536] nix-daemon[2834]: accepted connection from pid 2877, user nixbot1685gitlab # [ 210.728679] nix-daemon[2834]: accepted connection from pid 2829, user nixbot1686gitlab # [ 215.078654] nix-daemon[2834]: accepted connection from pid 2829, user nixbot1687gitlab # [ 215.179812] nix-daemon[2834]: accepted connection from pid 2926, user nixbot1688gitlab # [ 215.211920] gitaly[917]: time="2026-09-06T16:34:03.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3QCBG0S81G3GYAH63HED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:34:33.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:03.320" grpc.time_ms=0.739 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689gitlab # [ 215.261358] gitaly[917]: time="2026-09-06T16:34:03.371Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3QCBG0S81G3GYAH63HED grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:13.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:03.360" grpc.time_ms=10.837 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690gitlab # [ 215.297090] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1301691gitlab # [ 215.298926] nixbot[2660]: {"ts": "2026-09-06T16:34:03.407Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1692gitlab # [ 215.353883] gitaly[917]: time="2026-09-06T16:34:03.464Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3QGS126JNBCJEWH8W8Y4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:34:33.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:03.462" grpc.time_ms=1.004 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693gitlab # [ 215.460763] gitaly[917]: time="2026-09-06T16:34:03.570Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3QGS126JNBCJEWH8W8Y4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:13.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:03.561" grpc.time_ms=9.098 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1694gitlab # [ 215.681370] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3731695gitlab # [ 215.684562] nixbot[2660]: {"ts": "2026-09-06T16:34:03.794Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1696gitlab # [ 220.310865] nix-daemon[2834]: accepted connection from pid 2993, user nixbot1697gitlab # [ 220.376975] gitaly[917]: time="2026-09-06T16:34:08.486Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3WDWC3Q9105G3ESVPK5P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:34:38.184" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:08.485" grpc.time_ms=0.74 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1698gitlab # [ 220.401528] gitaly[917]: time="2026-09-06T16:34:08.511Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3WDWC3Q9105G3ESVPK5P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:18.102" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:08.502" grpc.time_ms=8.6 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1699gitlab # [ 220.470224] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1351700gitlab # [ 220.471721] nixbot[2660]: {"ts": "2026-09-06T16:34:08.581Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1701gitlab # [ 220.790744] gitaly[917]: time="2026-09-06T16:34:08.900Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3WTVA2BGGJ9CNQDVDCJA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:08.892" grpc.time_ms=8.155 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1702gitlab # [ 220.818875] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691703gitlab # [ 220.893094] gitaly[917]: time="2026-09-06T16:34:09.003Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3WX7XF3TNKPNNTB35SSM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:08.990" pid=917 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1704gitlab # [ 220.894737] gitaly[917]: time="2026-09-06T16:34:09.003Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3WX7XF3TNKPNNTB35SSM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:08.990" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1705gitlab # [ 220.938746] gitaly[917]: time="2026-09-06T16:34:09.048Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3WX7XF3TNKPNNTB35SSM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:08.990" grpc.time_ms=58.125 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1706gitlab # [ 220.951601] gitlab-workhorse[826]: time="2026-09-06T16:34:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671707gitlab # [ 220.952805] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1261708gitlab # [ 221.036992] gitaly[917]: time="2026-09-06T16:34:09.147Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3X2ASB4TWWCQTXCRABXX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:09.136" grpc.time_ms=9.676 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1709gitlab # [ 221.057984] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:09 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 691710gitlab # [ 221.116528] gitaly[917]: time="2026-09-06T16:34:09.226Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3X4J20VK25A4ZK8TBXDE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:09.219" pid=917 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1711gitlab # [ 221.137698] gitaly[917]: time="2026-09-06T16:34:09.226Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3X4J20VK25A4ZK8TBXDE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:09.219" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1712gitlab # [ 221.158331] gitaly[917]: time="2026-09-06T16:34:09.268Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3X4J20VK25A4ZK8TBXDE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:09.219" grpc.time_ms=48.976 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1713gitlab # [ 221.158868] gitlab-workhorse[826]: time="2026-09-06T16:34:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671714gitlab # [ 221.159433] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 981715gitlab # [ 221.385435] sidekiq-cluster[2544]: 2026-09-06T16:34:09.494Z pid=2544 tid=xug INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2544, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430891008}1716gitlab # [ 221.561182] sidekiq-cluster[2544]: W, [2026-09-06T16:34:09.671232 #2544] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1717gitlab # [ 221.601170] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:09.711Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1718gitlab # [ 223.321413] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:11.430Z","message":"GitLab reliable fetch activated!"}1719gitlab # [ 223.322669] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:11.430Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1720gitlab # [ 242.266330] postgres[3226]: [3226] ERROR: relation "table_without_models" does not exist1721gitlab # [ 242.268663] postgres[3226]: [3226] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1722gitlab # [ 244.624622] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:32.733Z","message":"Booted Rails 7.2.3.1 application in production environment"}1723gitlab # [ 244.627401] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:32.736Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1724gitlab # [ 244.628462] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:32.736Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1725gitlab # [ 244.629956] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:32.736Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1726gitlab # [ 244.631579] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:32.737Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1727gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 41.42 seconds)1728gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1729gitlab # [ 245.257314] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:33 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 401730gitlab # [ 245.354832] gitaly[917]: time="2026-09-06T16:34:33.464Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MRW3B2K2Z5H925YJ55B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:33.452" grpc.time_ms=11.918 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1731gitlab # [ 245.356390] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:33 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 911732gitlab # Enumerating objects: 5, done.1733gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1734gitlab # Delta compression using up to 4 threads1735gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1736gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1737gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1738gitlab # [ 245.590433] gitaly[917]: time="2026-09-06T16:34:33.700Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=332 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:35:03.193" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:34:33.694" grpc.time_ms=6.071 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab # [ 245.604130] gitaly[917]: time="2026-09-06T16:34:33.714Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VS4MWYDTC9Q9D31N44V00V duration_ms=66ns method=POST pid=917 status=200 url="http://unix/api/v4/internal/allowed"1740gitlab # [ 245.635394] gitaly[917]: time="2026-09-06T16:34:33.745Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VS4MWYDTC9Q9D31N44V00V duration_ms=30ns method=POST pid=917 status=200 url="http://unix/api/v4/internal/pre_receive"1741gitlab # [ 245.636355] gitaly[917]: time="2026-09-06T16:34:33.746Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5308 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:33.647" grpc.time_ms=98.69 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1742gitlab # [ 245.685579] gitaly[917]: time="2026-09-06T16:34:33.795Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:33.795" grpc.time_ms=0.524 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1743gitlab # [ 245.723118] gitaly[917]: time="2026-09-06T16:34:33.833Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4777 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:33.832" grpc.time_ms=0.72 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1744gitlab # [ 245.766526] gitaly[917]: time="2026-09-06T16:34:33.876Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:33.876" grpc.time_ms=0.394 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1745gitlab # [ 245.868288] gitaly[917]: time="2026-09-06T16:34:33.978Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VS4MWYDTC9Q9D31N44V00V duration_ms=63ns method=POST pid=917 status=200 url="http://unix/api/v4/internal/post_receive"1746gitlab # [ 245.875379] gitaly[917]: time="2026-09-06T16:34:33.985Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4802 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:33.913" grpc.time_ms=71.507 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1747gitlab # [ 245.908091] gitaly[917]: time="2026-09-06T16:34:34.018Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V diskcache=03122b9a-e85a-4cf6-a19f-345d6d99b7a3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:33.574" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1748gitlab # [ 245.925421] gitaly[917]: time="2026-09-06T16:34:34.035Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=188 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=14021 command.oublock=32 command.real_time_ms=434 command.spawn_token_fork_ms=0 command.system_time_ms=127 command.user_time_ms=61 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:33.574" grpc.time_ms=444.39 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1749gitlab # [ 245.926650] gitlab-workhorse[826]: time="2026-09-06T16:34:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521750gitlab # [ 245.927605] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:34 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 5321751gitlab # To http://gitlab/root/test-flake.git1752gitlab # e485f2c..0984b3f master -> master1753gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.78 seconds)1754gitlab: must succeed: git -C /tmp/test-flake rev-parse master1755gitlab # [ 245.947966] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.057Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5404dd8e6ce10ad7a23aa720","created_at":"2026-09-06T16:33:41.924Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:41.957Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5404dd8e6ce10ad7a23aa720: start","job_status":"start","queue_duration_s":52.099749,"scheduling_latency_s":52.099797}1756gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1757gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1758gitlab # [ 246.037683] nixbot[2660]: INFO: ::ffff:127.0.0.1:60156 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1759gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.08 seconds)1760gitlab # [ 246.100372] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.209Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"5404dd8e6ce10ad7a23aa720","created_at":"2026-09-06T16:33:41.924Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:41.957Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-5404dd8e6ce10ad7a23aa720: done: 0.152877 sec","job_status":"done","queue_duration_s":52.099749,"scheduling_latency_s":52.099797,"redis_calls":25,"redis_duration_s":0.005778,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002827,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001707,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.067866447,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.152877,"completed_at":"2026-09-06T16:34:34.209Z","load_balancing_strategy":"primary","db_duration_s":0.034912,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1761gitlab # [ 246.169088] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.278Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"e5e1d74a71c20073ab422229","created_at":"2026-09-06T16:33:41.967Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:41.971Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-e5e1d74a71c20073ab422229: start","job_status":"start","queue_duration_s":52.306799,"scheduling_latency_s":52.306851}1762gitlab # [ 246.289560] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.399Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"e5e1d74a71c20073ab422229","created_at":"2026-09-06T16:33:41.967Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:41.971Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-e5e1d74a71c20073ab422229: done: 0.120815 sec","job_status":"done","queue_duration_s":52.306799,"scheduling_latency_s":52.306851,"redis_calls":14,"redis_duration_s":0.003434,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000651,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082781472,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.120815,"completed_at":"2026-09-06T16:34:34.399Z","load_balancing_strategy":"primary","db_duration_s":0.025322,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1763gitlab # [ 246.363106] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.472Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"82017e220ee2cd43f84844ad","created_at":"2026-09-06T16:33:42.487Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:42.492Z","job_size_bytes":5,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-82017e220ee2cd43f84844ad: start","job_status":"start","queue_duration_s":51.980592,"scheduling_latency_s":51.980643}1764gitlab # [ 246.525468] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.635Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"82017e220ee2cd43f84844ad","created_at":"2026-09-06T16:33:42.487Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:42.492Z","job_size_bytes":5,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-82017e220ee2cd43f84844ad: done: 0.162556 sec","job_status":"done","queue_duration_s":51.980592,"scheduling_latency_s":51.980643,"redis_calls":12,"redis_duration_s":0.002843,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000664,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001713,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.111422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000307581,"duration_s":0.162556,"completed_at":"2026-09-06T16:34:34.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028137,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1765gitlab # [ 246.602978] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.712Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"813d315c353c7706a7249614","created_at":"2026-09-06T16:33:42.509Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:42.511Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-813d315c353c7706a7249614: start","job_status":"start","queue_duration_s":52.200812,"scheduling_latency_s":52.200865}1766gitlab # [ 246.806827] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.916Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"813d315c353c7706a7249614","created_at":"2026-09-06T16:33:42.509Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:42.511Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-813d315c353c7706a7249614: done: 0.204103 sec","job_status":"done","queue_duration_s":52.200812,"scheduling_latency_s":52.200865,"redis_calls":11,"redis_duration_s":0.002999,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00069,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.148,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.148,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000645333,"duration_s":0.204103,"completed_at":"2026-09-06T16:34:34.916Z","load_balancing_strategy":"primary","db_duration_s":0.072782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1767gitlab # [ 246.883668] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:34.993Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f1ae02281e2cdeb119506349","created_at":"2026-09-06T16:33:42.514Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:42.516Z","job_size_bytes":89,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f1ae02281e2cdeb119506349: start","job_status":"start","queue_duration_s":52.476618,"scheduling_latency_s":52.476677}1768gitlab # [ 246.949528] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:35.058Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"f1ae02281e2cdeb119506349","created_at":"2026-09-06T16:33:42.514Z","trace_propagation_headers":{"sentry-trace":"afbd87a756c84500894159cc45cda330-d38a022c35a144b2","baggage":"sentry-trace_id=afbd87a756c84500894159cc45cda330,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS31R09QBJC2M2X488RC4N","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:42.516Z","job_size_bytes":89,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-f1ae02281e2cdeb119506349: done: 0.065077 sec","job_status":"done","queue_duration_s":52.476618,"scheduling_latency_s":52.476677,"redis_calls":11,"redis_duration_s":0.002779,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000703,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000476038,"duration_s":0.065077,"completed_at":"2026-09-06T16:34:35.058Z","load_balancing_strategy":"primary","db_duration_s":0.00725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1769gitlab # [ 247.010077] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:35.119Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 e485f2cbba70e6f175cf8098620ece9de959d0ad refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"072a2d6f5676c0ccb4d64524","created_at":"2026-09-06T16:33:49.041Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-47ce79731c5e4424","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:49.046Z","job_size_bytes":185,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-072a2d6f5676c0ccb4d64524: start","job_status":"start","queue_duration_s":46.073673,"scheduling_latency_s":46.073713}1770gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1771gitlab # [ 247.045731] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:35.155Z","message":"changes: \"0000000000000000000000000000000000000000 e485f2cbba70e6f175cf8098620ece9de959d0ad refs/heads/master\\n\""}1772gitlab # [ 247.076894] nixbot[2660]: INFO: ::ffff:127.0.0.1:60166 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1773gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.04 seconds)1774gitlab # [ 247.100055] gitaly[917]: time="2026-09-06T16:34:35.209Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:45.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:35.199" grpc.time_ms=10.235 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 247.129849] gitaly[917]: time="2026-09-06T16:34:35.240Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:34:45.038" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:35.229" grpc.time_ms=10.761 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1776gitlab # [ 247.195878] gitaly[917]: time="2026-09-06T16:34:35.306Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:34:45.103" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:35.293" grpc.time_ms=12.347 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1777gitlab # [ 247.240745] gitaly[917]: time="2026-09-06T16:34:35.350Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3930 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:35.350" grpc.time_ms=0.782 pid=917 span.kind=server system=grpc1778gitlab # [ 247.290857] gitaly[917]: time="2026-09-06T16:34:35.401Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3932 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:35.400" grpc.time_ms=0.597 pid=917 span.kind=server system=grpc1779gitlab # [ 247.325883] gitaly[917]: time="2026-09-06T16:34:35.436Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 diskcache=1824fe70-a453-490e-bcde-9b85773296a3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:34:45.126" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:35.316" pid=917 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1780gitlab # [ 247.345167] gitaly[917]: time="2026-09-06T16:34:35.436Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=95 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4945 command.oublock=8 command.real_time_ms=109 command.spawn_token_fork_ms=0 command.system_time_ms=54 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:34:45.126" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:35.316" grpc.time_ms=119.82 pid=917 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1781gitlab # [ 247.345829] gitaly[917]: time="2026-09-06T16:34:35.450Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:34:45.259" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:35.449" grpc.time_ms=0.212 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782gitlab # [ 247.359725] gitaly[917]: time="2026-09-06T16:34:35.469Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:34:45.264" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:35.454" grpc.time_ms=14.846 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab # [ 247.426177] gitaly[917]: time="2026-09-06T16:34:35.536Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-06T16:35:05.028" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:35.528" grpc.time_ms=7.173 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1784gitlab # [ 247.460064] gitaly[917]: time="2026-09-06T16:34:35.570Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:34:45.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:35.569" grpc.time_ms=0.347 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1785gitlab # [ 247.469775] gitaly[917]: time="2026-09-06T16:34:35.579Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:34:45.012" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:34:35.572" grpc.time_ms=7.327 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786gitlab # [ 247.497152] gitaly[917]: time="2026-09-06T16:34:35.607Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:45.035" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:35.595" grpc.time_ms=11.091 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 247.515721] gitaly[917]: time="2026-09-06T16:34:35.625Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:45.055" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:35.615" grpc.time_ms=10.196 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1788gitlab # [ 247.547337] gitaly[917]: time="2026-09-06T16:34:35.657Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:35:05.139" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:35.639" grpc.time_ms=17.6 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 247.899823] gitaly[917]: time="2026-09-06T16:34:36.009Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:34:46.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:36.008" grpc.time_ms=1.49 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 247.927875] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.033Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 e485f2cbba70e6f175cf8098620ece9de959d0ad refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"072a2d6f5676c0ccb4d64524","created_at":"2026-09-06T16:33:49.041Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-47ce79731c5e4424","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:33:49.046Z","job_size_bytes":185,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-072a2d6f5676c0ccb4d64524: done: 0.913549 sec","job_status":"done","queue_duration_s":46.073673,"scheduling_latency_s":46.073713,"gitaly_calls":13,"gitaly_duration_s":0.300374,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.04618,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.004094,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006865,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001609,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00445,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018771,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010391,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.178,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.199,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.495048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001175848,"exclusive_lock_hold_duration_s":0.053323537,"duration_s":0.913549,"completed_at":"2026-09-06T16:34:36.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.094902,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1791gitlab # [ 247.990064] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.034Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5b2cc6ecea7f0a5cf7cc4b67","created_at":"2026-09-06T16:33:53.770Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.779Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5b2cc6ecea7f0a5cf7cc4b67: start","job_status":"start","queue_duration_s":42.25541,"scheduling_latency_s":42.255426}1792gitlab # [ 247.990787] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.096Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5b2cc6ecea7f0a5cf7cc4b67","created_at":"2026-09-06T16:33:53.770Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.779Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5b2cc6ecea7f0a5cf7cc4b67: done: 0.061897 sec","job_status":"done","queue_duration_s":42.25541,"scheduling_latency_s":42.255426,"redis_calls":16,"redis_duration_s":0.003778,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002014,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000323784,"duration_s":0.061897,"completed_at":"2026-09-06T16:34:36.096Z","load_balancing_strategy":"primary","db_duration_s":0.021233,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1793gitlab # [ 248.036569] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.097Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8f01a44a059947466966c093","created_at":"2026-09-06T16:33:53.781Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.785Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8f01a44a059947466966c093: start","job_status":"start","queue_duration_s":42.311996,"scheduling_latency_s":42.312012}1794gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1795gitlab # [ 248.116937] nixbot[2660]: INFO: ::ffff:127.0.0.1:60180 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1796gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.04 seconds)1797gitlab # [ 248.182226] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.291Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8f01a44a059947466966c093","created_at":"2026-09-06T16:33:53.781Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.785Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8f01a44a059947466966c093: done: 0.194025 sec","job_status":"done","queue_duration_s":42.311996,"scheduling_latency_s":42.312012,"redis_calls":21,"redis_duration_s":0.006416,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003157,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001656,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000432457,"duration_s":0.194025,"completed_at":"2026-09-06T16:34:36.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034571,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1798gitlab # [ 248.251668] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.361Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8494297b04257d8c1b3fc936","created_at":"2026-09-06T16:33:53.869Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.878Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8494297b04257d8c1b3fc936: start","job_status":"start","queue_duration_s":42.482959,"scheduling_latency_s":42.482993}1799gitlab # [ 248.456381] gitaly[917]: time="2026-09-06T16:34:36.565Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:36.565" grpc.time_ms=0.425 pid=917 span.kind=server system=grpc1800gitlab # [ 248.513225] gitaly[917]: time="2026-09-06T16:34:36.623Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3889 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:36.622" grpc.time_ms=0.67 pid=917 span.kind=server system=grpc1801gitlab # [ 248.563144] gitaly[917]: time="2026-09-06T16:34:36.673Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:36.672" grpc.time_ms=0.739 pid=917 span.kind=server system=grpc1802gitlab # [ 248.587395] gitaly[917]: time="2026-09-06T16:34:36.697Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F diskcache=96e17422-8a9e-44a9-a372-d3cbb91ce666 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:06.025" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:36.526" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab # [ 248.596830] gitaly[917]: time="2026-09-06T16:34:36.706Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=150 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7406 command.oublock=0 command.real_time_ms=161 command.spawn_token_fork_ms=0 command.system_time_ms=85 command.user_time_ms=65 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:06.025" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:36.526" grpc.time_ms=171.339 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 248.746317] gitaly[917]: time="2026-09-06T16:34:36.856Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:06.354" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:36.854" grpc.time_ms=1.427 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 248.784698] gitaly[917]: time="2026-09-06T16:34:36.894Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS3DNGSFMV5TPZTREVHB0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:06.392" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:36.893" grpc.time_ms=1.007 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 248.828366] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.938Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8494297b04257d8c1b3fc936","created_at":"2026-09-06T16:33:53.869Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.878Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2544,"job_status":"rescheduled","message":"PipelineProcessWorker JID-8494297b04257d8c1b3fc936: rescheduled"}1807gitlab # [ 248.871493] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:36.980Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8494297b04257d8c1b3fc936","created_at":"2026-09-06T16:33:53.869Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-1c204b92ce3143a1","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:53.878Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8494297b04257d8c1b3fc936: done: 0.619397 sec","job_status":"done","queue_duration_s":42.482959,"scheduling_latency_s":42.482993,"gitaly_calls":3,"gitaly_duration_s":0.229344,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.060669999999999995,"redis_read_bytes":460,"redis_write_bytes":29290,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001202,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001115,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.011531,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005222,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15577,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.036507,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000588,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000615,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002688,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.023,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.266289,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.523842555,"duration_s":0.619397,"completed_at":"2026-09-06T16:34:36.980Z","load_balancing_strategy":"primary","db_duration_s":0.072696,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1808gitlab # [ 248.934920] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.044Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","e485f2cbba70e6f175cf8098620ece9de959d0ad 0984b3f14362b0077b7998aa4bc82cc88650f4d3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ea8e2af144ec39a89fb1b81c","created_at":"2026-09-06T16:34:33.947Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d399f8a046a14def","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:33.949Z","job_size_bytes":185,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ea8e2af144ec39a89fb1b81c: start","job_status":"start","queue_duration_s":3.094687,"scheduling_latency_s":3.09474}1809gitlab # [ 248.954112] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.064Z","message":"changes: \"e485f2cbba70e6f175cf8098620ece9de959d0ad 0984b3f14362b0077b7998aa4bc82cc88650f4d3 refs/heads/master\\n\""}1810gitlab # [ 248.988213] gitaly[917]: time="2026-09-06T16:34:37.098Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:47.088" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:37.088" grpc.time_ms=9.24 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 249.003325] gitaly[917]: time="2026-09-06T16:34:37.113Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:34:47.102" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:37.103" grpc.time_ms=9.874 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 249.016840] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.118Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"eed5ba552d9593a1604234ee","created_at":"2026-09-06T16:34:37.116Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"fbf3b6fd4b4c0d9cf7a3f87e","job_size_bytes":46,"pid":2544,"job_status":"deduplicated","message":"ProjectCacheWorker JID-eed5ba552d9593a1604234ee: deduplicated: until executing","deduplication.type":"until executing"}1813gitlab # [ 249.027393] gitaly[917]: time="2026-09-06T16:34:37.137Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:34:47.128" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:37.129" grpc.time_ms=8.142 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 249.058322] gitaly[917]: time="2026-09-06T16:34:37.168Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-06T16:35:07.055" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.156" grpc.time_ms=12.075 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1815gitlab # [ 249.075661] gitaly[917]: time="2026-09-06T16:34:37.185Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:34:47.181" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:34:37.181" grpc.time_ms=3.757 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1816gitlab # [ 249.107728] gitaly[917]: time="2026-09-06T16:34:37.217Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:47.017" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:37.207" grpc.time_ms=9.941 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1818gitlab # [ 249.146824] nixbot[2660]: INFO: ::ffff:127.0.0.1:60186 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1819gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.03 seconds)1820gitlab # [ 249.207300] gitaly[917]: time="2026-09-06T16:34:37.317Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:34:47.124" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.315" grpc.time_ms=1.865 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 249.233892] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.343Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","e485f2cbba70e6f175cf8098620ece9de959d0ad 0984b3f14362b0077b7998aa4bc82cc88650f4d3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ea8e2af144ec39a89fb1b81c","created_at":"2026-09-06T16:34:33.947Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d399f8a046a14def","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:33.949Z","job_size_bytes":185,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ea8e2af144ec39a89fb1b81c: done: 0.299221 sec","job_status":"done","queue_duration_s":3.094687,"scheduling_latency_s":3.09474,"gitaly_calls":7,"gitaly_duration_s":0.101375,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.025866,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.002146,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000998,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003324,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.008432,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.010966,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.026,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.152804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008976,"exclusive_lock_hold_duration_s":0.02223439,"duration_s":0.299221,"completed_at":"2026-09-06T16:34:37.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02888,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1822gitlab # [ 249.281166] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.345Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"fbf3b6fd4b4c0d9cf7a3f87e","created_at":"2026-09-06T16:34:35.243Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.266Z","job_size_bytes":46,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-fbf3b6fd4b4c0d9cf7a3f87e: start","job_status":"start","queue_duration_s":2.078697,"scheduling_latency_s":2.078727}1823gitlab # [ 249.282158] gitaly[917]: time="2026-09-06T16:34:37.387Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-06T22:34:37.380" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:37.381" grpc.time_ms=5.233 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1824gitlab # [ 249.296069] gitaly[917]: time="2026-09-06T16:34:37.405Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:35:07.292" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.393" grpc.time_ms=12.065 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825gitlab # [ 249.316282] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.426Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"fbf3b6fd4b4c0d9cf7a3f87e","created_at":"2026-09-06T16:34:35.243Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.266Z","job_size_bytes":46,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-fbf3b6fd4b4c0d9cf7a3f87e: done: 0.080628 sec","job_status":"done","queue_duration_s":2.078697,"scheduling_latency_s":2.078727,"gitaly_calls":2,"gitaly_duration_s":0.022492,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006795,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001289,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000492,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001766,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002581,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000279924,"duration_s":0.080628,"completed_at":"2026-09-06T16:34:37.426Z","load_balancing_strategy":"primary","db_duration_s":0.012744,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1826gitlab # [ 249.362873] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.427Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"fe434477ebf4faa451eb997d","created_at":"2026-09-06T16:34:35.283Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.288Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-fe434477ebf4faa451eb997d: start","job_status":"start","queue_duration_s":2.138803,"scheduling_latency_s":2.138819}1827gitlab # [ 249.368598] gitaly[917]: time="2026-09-06T16:34:37.478Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 diskcache=6c28e3e0-5d29-4ce2-a45e-f9b6d14ec41c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:34:37.444" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.444" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828gitlab # [ 249.369748] gitaly[917]: time="2026-09-06T16:34:37.479Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1287 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:34:37.444" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.444" grpc.time_ms=34.209 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829gitlab # [ 249.421568] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.531Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"fe434477ebf4faa451eb997d","created_at":"2026-09-06T16:34:35.283Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.288Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-fe434477ebf4faa451eb997d: done: 0.103711 sec","job_status":"done","queue_duration_s":2.138803,"scheduling_latency_s":2.138819,"gitaly_calls":1,"gitaly_duration_s":0.038512,"redis_calls":15,"redis_duration_s":0.003769,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001521,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000603,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.081212557,"duration_s":0.103711,"completed_at":"2026-09-06T16:34:37.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1830gitlab # [ 249.423954] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.533Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e3e34bf94b1add3465a7649d","created_at":"2026-09-06T16:34:35.445Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.447Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e3e34bf94b1add3465a7649d: start","job_status":"start","queue_duration_s":2.085819,"scheduling_latency_s":2.085847}1831gitlab # [ 249.458922] gitaly[917]: time="2026-09-06T16:34:37.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:35:07.058" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.558" grpc.time_ms=9.41 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 249.464610] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.574Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e3e34bf94b1add3465a7649d","created_at":"2026-09-06T16:34:35.445Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.447Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e3e34bf94b1add3465a7649d: done: 0.041525 sec","job_status":"done","queue_duration_s":2.085819,"scheduling_latency_s":2.085847,"gitaly_calls":1,"gitaly_duration_s":0.011399,"redis_calls":16,"redis_duration_s":0.004939000000000001,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000182,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.00136,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041525,"completed_at":"2026-09-06T16:34:37.574Z","load_balancing_strategy":"primary","db_duration_s":0.00228,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1833gitlab # [ 249.493523] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:37.575Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"89b2f9b246118fb8b52bce61","created_at":"2026-09-06T16:34:35.891Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9cbc8e71d733616cd398b2ee29cb956c7580ab818b1b398a8af34616d2031131","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.895Z","job_size_bytes":467,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-89b2f9b246118fb8b52bce61: start","job_status":"start","queue_duration_s":1.680581,"scheduling_latency_s":1.680599}1834gitlab # [ 249.545792] gitaly[917]: time="2026-09-06T16:34:37.591Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:07.090" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.591" grpc.time_ms=0.477 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab # [ 249.546586] gitaly[917]: time="2026-09-06T16:34:37.599Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:34:47.032" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:37.593" grpc.time_ms=5.773 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836gitlab # [ 249.563408] gitaly[917]: time="2026-09-06T16:34:37.673Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:07.172" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.672" grpc.time_ms=0.819 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 249.568922] gitaly[917]: time="2026-09-06T16:34:37.679Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=273 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:35:07.175" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:37.675" grpc.time_ms=3.165 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 249.626441] gitaly[917]: time="2026-09-06T16:34:37.736Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:47.166" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:37.727" grpc.time_ms=9.315 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 250.042531] gitaly[917]: time="2026-09-06T16:34:38.152Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:35:08.032" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.133" grpc.time_ms=19.254 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 250.095326] gitaly[917]: time="2026-09-06T16:34:38.205Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS38T6FT6GG0W5MPFD4J36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:35:08.087" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.187" grpc.time_ms=17.522 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1842gitlab # [ 250.159662] sidekiq-cluster[2544]: {"severity":"WARN","time":"2026-09-06T16:34:38.268Z","class":"CreatePipelineWorker","correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"e485f2cbba70e6f175cf8098620ece9de959d0ad","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e485f2cbba70e6f175cf8098620ece9de959d0ad","ref":"refs/heads/master"},"retry":0}1843gitlab # [ 250.188158] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"89b2f9b246118fb8b52bce61","created_at":"2026-09-06T16:34:35.891Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9cbc8e71d733616cd398b2ee29cb956c7580ab818b1b398a8af34616d2031131","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.895Z","job_size_bytes":467,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-89b2f9b246118fb8b52bce61: done: 0.721602 sec","job_status":"done","queue_duration_s":1.680581,"scheduling_latency_s":1.680599,"gitaly_calls":7,"gitaly_duration_s":0.128393,"redis_calls":51,"redis_duration_s":0.02404,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000548,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003597,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001616,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.526166,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000548394,"duration_s":0.721602,"completed_at":"2026-09-06T16:34:38.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037792,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1844gitlab # [ 250.198502] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.300Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"0e09b78e-0bc7-436e-b7f5-82c2f1e49315\"}"],"class":"WebHookWorker","jid":"fda962c168697b3ed5c421de","created_at":"2026-09-06T16:34:35.984Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4d46ca5c852177595760c6cd73fb74e1f71ffcadfeb9b894f843638e05b04d1a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.989Z","job_size_bytes":1751,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-fda962c168697b3ed5c421de: start","job_status":"start","queue_duration_s":2.31122,"scheduling_latency_s":2.311253}1845gitlab # [ 250.268739] nixbot[2660]: INFO: ::ffff:127.0.0.1:60202 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1846gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.12 seconds)1847gitlab # [ 250.350620] nixbot[2660]: INFO: ::ffff:127.0.0.1:60212 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1848gitlab # [ 250.373850] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.483Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"0e09b78e-0bc7-436e-b7f5-82c2f1e49315\"}"],"class":"WebHookWorker","jid":"fda962c168697b3ed5c421de","created_at":"2026-09-06T16:34:35.984Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-77896b272f704ba9","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4d46ca5c852177595760c6cd73fb74e1f71ffcadfeb9b894f843638e05b04d1a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:35.989Z","job_size_bytes":1751,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-fda962c168697b3ed5c421de: done: 0.183113 sec","job_status":"done","queue_duration_s":2.31122,"scheduling_latency_s":2.311253,"redis_calls":19,"redis_duration_s":0.005529,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0013,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000484,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002368,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001377,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.03271980099998473,"cpu_s":0.102961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398934,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.183113,"completed_at":"2026-09-06T16:34:38.483Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1849gitlab # [ 250.390249] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.487Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"73ebf226a55841c4f9aeb09b","created_at":"2026-09-06T16:34:36.514Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.517Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-73ebf226a55841c4f9aeb09b: start","job_status":"start","queue_duration_s":1.969502,"scheduling_latency_s":1.969535}1850gitlab # [ 250.450870] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.560Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"73ebf226a55841c4f9aeb09b","created_at":"2026-09-06T16:34:36.514Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.517Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-73ebf226a55841c4f9aeb09b: done: 0.073583 sec","job_status":"done","queue_duration_s":1.969502,"scheduling_latency_s":1.969535,"redis_calls":10,"redis_duration_s":0.001807,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000312,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001169,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000232152,"duration_s":0.073583,"completed_at":"2026-09-06T16:34:38.560Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01988,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1851gitlab # [ 250.457090] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.561Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cf2af060d78a7d90459cb28f","created_at":"2026-09-06T16:34:36.725Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.731Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cf2af060d78a7d90459cb28f: start","job_status":"start","queue_duration_s":1.829951,"scheduling_latency_s":1.829965}1852gitlab # [ 250.501895] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cf2af060d78a7d90459cb28f","created_at":"2026-09-06T16:34:36.725Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.731Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cf2af060d78a7d90459cb28f: done: 0.023281 sec","job_status":"done","queue_duration_s":1.829951,"scheduling_latency_s":1.829965,"redis_calls":10,"redis_duration_s":0.001304,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022433,"duration_s":0.023281,"completed_at":"2026-09-06T16:34:38.585Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004347,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853gitlab # [ 250.546838] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.586Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"71efff9bf60f2edf6cbfa85b","created_at":"2026-09-06T16:34:36.747Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.753Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-71efff9bf60f2edf6cbfa85b: start","job_status":"start","queue_duration_s":1.832515,"scheduling_latency_s":1.832529}1854gitlab # [ 250.547826] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"71efff9bf60f2edf6cbfa85b","created_at":"2026-09-06T16:34:36.747Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.753Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-71efff9bf60f2edf6cbfa85b: done: 0.020944 sec","job_status":"done","queue_duration_s":1.832515,"scheduling_latency_s":1.832529,"redis_calls":10,"redis_duration_s":0.001282,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213714,"duration_s":0.020944,"completed_at":"2026-09-06T16:34:38.607Z","load_balancing_strategy":"primary","db_duration_s":0.003055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1855gitlab # [ 250.599689] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.608Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"45ce59caf867d4a9b6a47268","created_at":"2026-09-06T16:34:36.768Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.773Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-45ce59caf867d4a9b6a47268: start","job_status":"start","queue_duration_s":1.834827,"scheduling_latency_s":1.834842}1856gitlab # [ 250.600445] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.632Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"45ce59caf867d4a9b6a47268","created_at":"2026-09-06T16:34:36.768Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.773Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-45ce59caf867d4a9b6a47268: done: 0.024867 sec","job_status":"done","queue_duration_s":1.834827,"scheduling_latency_s":1.834842,"redis_calls":13,"redis_duration_s":0.0016539999999999999,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000367,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226844,"duration_s":0.024867,"completed_at":"2026-09-06T16:34:38.632Z","load_balancing_strategy":"primary","db_duration_s":0.005307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1857gitlab # [ 250.649065] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.633Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ff13355d840b487813b3bc72","created_at":"2026-09-06T16:34:36.773Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.778Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ff13355d840b487813b3bc72: start","job_status":"start","queue_duration_s":1.85564,"scheduling_latency_s":1.855654}1858gitlab # [ 250.649840] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.653Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ff13355d840b487813b3bc72","created_at":"2026-09-06T16:34:36.773Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.778Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ff13355d840b487813b3bc72: done: 0.019292 sec","job_status":"done","queue_duration_s":1.85564,"scheduling_latency_s":1.855654,"redis_calls":10,"redis_duration_s":0.0012749999999999999,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.019292,"completed_at":"2026-09-06T16:34:38.653Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859gitlab # [ 250.673059] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.654Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d28cf91b869f1fc1060fa9c8","created_at":"2026-09-06T16:34:36.778Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.783Z","job_size_bytes":19,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d28cf91b869f1fc1060fa9c8: start","job_status":"start","queue_duration_s":1.870073,"scheduling_latency_s":1.870088}1860gitlab # [ 250.716667] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.673Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d28cf91b869f1fc1060fa9c8","created_at":"2026-09-06T16:34:36.778Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.783Z","job_size_bytes":19,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d28cf91b869f1fc1060fa9c8: done: 0.019786 sec","job_status":"done","queue_duration_s":1.870073,"scheduling_latency_s":1.870088,"redis_calls":11,"redis_duration_s":0.001375,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195835,"duration_s":0.019786,"completed_at":"2026-09-06T16:34:38.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861gitlab # [ 250.756716] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.674Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"15aec771244ef9f7e124f00b","created_at":"2026-09-06T16:34:36.785Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.793Z","job_size_bytes":66,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-15aec771244ef9f7e124f00b: start","job_status":"start","queue_duration_s":1.881403,"scheduling_latency_s":1.881418}1862gitlab # [ 250.757629] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.770Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"15aec771244ef9f7e124f00b","created_at":"2026-09-06T16:34:36.785Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.793Z","job_size_bytes":66,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-15aec771244ef9f7e124f00b: done: 0.095786 sec","job_status":"done","queue_duration_s":1.881403,"scheduling_latency_s":1.881418,"redis_calls":10,"redis_duration_s":0.001517,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.073,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025059,"duration_s":0.095786,"completed_at":"2026-09-06T16:34:38.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1863gitlab # [ 250.799754] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.771Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8eaed7ab911d3d0449160e5b","created_at":"2026-09-06T16:34:36.794Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.801Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8eaed7ab911d3d0449160e5b: start","job_status":"start","queue_duration_s":1.969988,"scheduling_latency_s":1.970003}1864gitlab # [ 250.800669] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.800Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8eaed7ab911d3d0449160e5b","created_at":"2026-09-06T16:34:36.794Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.801Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8eaed7ab911d3d0449160e5b: done: 0.028873 sec","job_status":"done","queue_duration_s":1.969988,"scheduling_latency_s":1.970003,"redis_calls":10,"redis_duration_s":0.001489,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000331,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000234667,"duration_s":0.028873,"completed_at":"2026-09-06T16:34:38.800Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003897,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865gitlab # [ 250.845150] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.801Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"5aa942398345a15f62b64b95","created_at":"2026-09-06T16:34:36.810Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.816Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-5aa942398345a15f62b64b95: start","job_status":"start","queue_duration_s":1.985469,"scheduling_latency_s":1.985485}1866gitlab # [ 250.885677] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.824Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"5aa942398345a15f62b64b95","created_at":"2026-09-06T16:34:36.810Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.816Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-5aa942398345a15f62b64b95: done: 0.02288 sec","job_status":"done","queue_duration_s":1.985469,"scheduling_latency_s":1.985485,"redis_calls":10,"redis_duration_s":0.0012799999999999999,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218743,"duration_s":0.02288,"completed_at":"2026-09-06T16:34:38.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867gitlab # [ 250.924483] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.825Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"69241c9e2297e3bf3d2a3be7","created_at":"2026-09-06T16:34:36.816Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.822Z","job_size_bytes":6,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-69241c9e2297e3bf3d2a3be7: start","job_status":"start","queue_duration_s":2.002937,"scheduling_latency_s":2.002951}1868gitlab # [ 250.925549] gitaly[917]: time="2026-09-06T16:34:38.937Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V diskcache=6ddff304-cdea-450e-ba0b-54813fdc031e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:34:38.926" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.927" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869gitlab # [ 250.926154] gitaly[917]: time="2026-09-06T16:34:38.937Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:34:38.926" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.927" grpc.time_ms=10.03 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870gitlab # [ 250.926815] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.838Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"69241c9e2297e3bf3d2a3be7","created_at":"2026-09-06T16:34:36.816Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.822Z","job_size_bytes":6,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-69241c9e2297e3bf3d2a3be7: done: 0.013396 sec","job_status":"done","queue_duration_s":2.002937,"scheduling_latency_s":2.002951,"redis_calls":10,"redis_duration_s":0.0013599999999999999,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218184,"duration_s":0.013396,"completed_at":"2026-09-06T16:34:38.838Z","load_balancing_strategy":"primary","db_duration_s":0.001307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1871gitlab # [ 250.965067] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.839Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"142ac3d0cf1156572fe3157b","created_at":"2026-09-06T16:34:36.823Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.828Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-142ac3d0cf1156572fe3157b: start","job_status":"start","queue_duration_s":2.010902,"scheduling_latency_s":2.010914}1872gitlab # [ 251.005736] gitaly[917]: time="2026-09-06T16:34:38.961Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:08.460" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.960" grpc.time_ms=0.55 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 251.006628] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"142ac3d0cf1156572fe3157b","created_at":"2026-09-06T16:34:36.823Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.828Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-142ac3d0cf1156572fe3157b: done: 0.020466 sec","job_status":"done","queue_duration_s":2.010902,"scheduling_latency_s":2.010914,"redis_calls":10,"redis_duration_s":0.0013419999999999999,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200305,"duration_s":0.020466,"completed_at":"2026-09-06T16:34:38.860Z","load_balancing_strategy":"primary","db_duration_s":0.00375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1874gitlab # [ 251.027973] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.861Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"e186a774e498e6be931be7c5","created_at":"2026-09-06T16:34:36.835Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.841Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-e186a774e498e6be931be7c5: start","job_status":"start","queue_duration_s":2.019343,"scheduling_latency_s":2.019357}1875gitlab # [ 251.073602] gitaly[917]: time="2026-09-06T16:34:38.970Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:34:48.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:38.963" grpc.time_ms=5.946 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 251.074415] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.898Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"e186a774e498e6be931be7c5","created_at":"2026-09-06T16:34:36.835Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.841Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-e186a774e498e6be931be7c5: done: 0.037942 sec","job_status":"done","queue_duration_s":2.019343,"scheduling_latency_s":2.019357,"redis_calls":14,"redis_duration_s":0.0034019999999999996,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000835,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241092,"duration_s":0.037942,"completed_at":"2026-09-06T16:34:38.898Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1877gitlab # [ 251.093158] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.899Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8384d132e065a870dfc39bca","created_at":"2026-09-06T16:34:36.938Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.970Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8384d132e065a870dfc39bca: start","job_status":"start","queue_duration_s":1.929139,"scheduling_latency_s":1.929153}1878gitlab # [ 251.137369] gitaly[917]: time="2026-09-06T16:34:38.990Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:08.889" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.989" grpc.time_ms=0.572 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 251.137992] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.915Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8384d132e065a870dfc39bca","created_at":"2026-09-06T16:34:36.938Z","trace_propagation_headers":{"sentry-trace":"db0bfc205ced4249b59b5b129b08aa9a-4365de5733d24b50","baggage":"sentry-trace_id=db0bfc205ced4249b59b5b129b08aa9a,sentry-release=Unknown"},"correlation_id":"01M1VS3DNGSFMV5TPZTREVHB0F","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:36.970Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8384d132e065a870dfc39bca: done: 0.015742 sec","job_status":"done","queue_duration_s":1.929139,"scheduling_latency_s":1.929153,"redis_calls":8,"redis_duration_s":0.001123,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015742,"completed_at":"2026-09-06T16:34:38.915Z","load_balancing_strategy":"primary","db_duration_s":0.002356,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1880gitlab # [ 251.178590] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.916Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"28fae466a4c60c57143c2f28","created_at":"2026-09-06T16:34:37.123Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"fe434477ebf4faa451eb997d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.125Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-28fae466a4c60c57143c2f28: start","job_status":"start","queue_duration_s":1.791178,"scheduling_latency_s":1.791191}1881gitlab # [ 251.179515] gitaly[917]: time="2026-09-06T16:34:38.993Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=75 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:35:08.891" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:38.991" grpc.time_ms=1.749 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882gitlab # [ 251.183715] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.946Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"28fae466a4c60c57143c2f28","created_at":"2026-09-06T16:34:37.123Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"fe434477ebf4faa451eb997d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.125Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-28fae466a4c60c57143c2f28: done: 0.030027 sec","job_status":"done","queue_duration_s":1.791178,"scheduling_latency_s":1.791191,"gitaly_calls":1,"gitaly_duration_s":0.011969,"redis_calls":12,"redis_duration_s":0.001781,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000388,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000647,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.019869285,"duration_s":0.030027,"completed_at":"2026-09-06T16:34:38.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1883gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1884gitlab # [ 251.236976] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:38.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8493ef0a3e632a1b2ce8bd2e","created_at":"2026-09-06T16:34:37.275Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2343c00fe177f08b2b9a440114955c6c10e2315bb3378c813fb3a5a0309d514e","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.278Z","job_size_bytes":467,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8493ef0a3e632a1b2ce8bd2e: start","job_status":"start","queue_duration_s":1.669646,"scheduling_latency_s":1.66966}1885gitlab # [ 251.239163] gitaly[917]: time="2026-09-06T16:34:39.013Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:49.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:39.007" grpc.time_ms=5.579 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886gitlab # [ 251.282586] gitaly[917]: time="2026-09-06T16:34:39.363Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:35:09.252" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:39.352" grpc.time_ms=10.728 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 251.283432] sidekiq-cluster[2544]: {"severity":"WARN","time":"2026-09-06T16:34:39.382Z","class":"CreatePipelineWorker","correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"e485f2cbba70e6f175cf8098620ece9de959d0ad","after":"0984b3f14362b0077b7998aa4bc82cc88650f4d3","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"0984b3f14362b0077b7998aa4bc82cc88650f4d3","ref":"refs/heads/master"},"retry":0}1888gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.04 seconds)1889gitlab # [ 251.283984] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.386Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8493ef0a3e632a1b2ce8bd2e","created_at":"2026-09-06T16:34:37.275Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2343c00fe177f08b2b9a440114955c6c10e2315bb3378c813fb3a5a0309d514e","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.278Z","job_size_bytes":467,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8493ef0a3e632a1b2ce8bd2e: done: 0.438581 sec","job_status":"done","queue_duration_s":1.669646,"scheduling_latency_s":1.66966,"gitaly_calls":7,"gitaly_duration_s":0.043167,"redis_calls":33,"redis_duration_s":0.003329,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000368,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001562,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.383507,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.438581,"completed_at":"2026-09-06T16:34:39.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010349,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1890gitlab # [ 251.327656] gitaly[917]: time="2026-09-06T16:34:39.373Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4MWYDTC9Q9D31N44V00V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:35:09.266" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:39.366" grpc.time_ms=6.49 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 251.328558] nixbot[2660]: INFO: ::ffff:127.0.0.1:60220 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1892gitlab # [ 251.328958] nixbot[2660]: INFO: ::ffff:127.0.0.1:60236 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1893gitlab # [ 251.329711] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.387Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"515c8c69-40a5-4ce6-bce0-6b652259cb63\"}"],"class":"WebHookWorker","jid":"1426c2ef59ae5ba62727d04f","created_at":"2026-09-06T16:34:37.295Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:45a5a0e84d5194f1cf584ff7ffa47640cdf8ab7cc086ea12df8822dc30b4c9cf","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.298Z","job_size_bytes":1749,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-1426c2ef59ae5ba62727d04f: start","job_status":"start","queue_duration_s":2.089177,"scheduling_latency_s":2.089195}1894gitlab # [ 251.371738] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.419Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"515c8c69-40a5-4ce6-bce0-6b652259cb63\"}"],"class":"WebHookWorker","jid":"1426c2ef59ae5ba62727d04f","created_at":"2026-09-06T16:34:37.295Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-d59d9c4790ba4678","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:45a5a0e84d5194f1cf584ff7ffa47640cdf8ab7cc086ea12df8822dc30b4c9cf","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.298Z","job_size_bytes":1749,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-1426c2ef59ae5ba62727d04f: done: 0.031556 sec","job_status":"done","queue_duration_s":2.089177,"scheduling_latency_s":2.089195,"redis_calls":14,"redis_duration_s":0.001767,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.00021,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007933690000015758,"cpu_s":0.018007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.031556,"completed_at":"2026-09-06T16:34:39.418Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1895gitlab # [ 251.418296] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.420Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e65c982916c1c7d32c3bee78","created_at":"2026-09-06T16:34:37.415Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-6182cb5cb46547d8","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.418Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e65c982916c1c7d32c3bee78: start","job_status":"start","queue_duration_s":2.00179,"scheduling_latency_s":2.001812}1896gitlab # [ 251.418576] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.451Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"e65c982916c1c7d32c3bee78","created_at":"2026-09-06T16:34:37.415Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-6182cb5cb46547d8","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:37.418Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-e65c982916c1c7d32c3bee78: done: 0.030625 sec","job_status":"done","queue_duration_s":2.00179,"scheduling_latency_s":2.001812,"redis_calls":19,"redis_duration_s":0.002541,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00035,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004905373,"duration_s":0.030625,"completed_at":"2026-09-06T16:34:39.450Z","load_balancing_strategy":"primary","db_duration_s":0.004723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1897gitlab # [ 251.459892] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.452Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","fda962c168697b3ed5c421de"],"class":"WebHooks::LogExecutionWorker","jid":"04e473f94dc6d0e9feba4d4a","created_at":"2026-09-06T16:34:38.464Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-a7100453079c46c3","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:de336d9ad75cf1303284510d1eefcaf88c8787fafca579fd73d39f96db276210","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:38.472Z","job_size_bytes":2593,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-04e473f94dc6d0e9feba4d4a: start","job_status":"start","queue_duration_s":0.980224,"scheduling_latency_s":0.980239}1898gitlab # [ 251.460722] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.480Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","fda962c168697b3ed5c421de"],"class":"WebHooks::LogExecutionWorker","jid":"04e473f94dc6d0e9feba4d4a","created_at":"2026-09-06T16:34:38.464Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-a7100453079c46c3","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:de336d9ad75cf1303284510d1eefcaf88c8787fafca579fd73d39f96db276210","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:38.472Z","job_size_bytes":2593,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-04e473f94dc6d0e9feba4d4a: done: 0.028367 sec","job_status":"done","queue_duration_s":0.980224,"scheduling_latency_s":0.980239,"redis_calls":10,"redis_duration_s":0.001323,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000211479,"duration_s":0.028367,"completed_at":"2026-09-06T16:34:39.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1899gitlab # [ 251.529753] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.481Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","1426c2ef59ae5ba62727d04f"],"class":"WebHooks::LogExecutionWorker","jid":"000244073b1a841032d71328","created_at":"2026-09-06T16:34:39.411Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-96e2b746be4b43ef","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dc59d8dcc99b2ad20e06a3d92745079e9ef2965b67a570211185faae6c17e7c2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:39.413Z","job_size_bytes":2592,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-000244073b1a841032d71328: start","job_status":"start","queue_duration_s":0.068227,"scheduling_latency_s":0.06824}1900gitlab # [ 251.530543] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.498Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","1426c2ef59ae5ba62727d04f"],"class":"WebHooks::LogExecutionWorker","jid":"000244073b1a841032d71328","created_at":"2026-09-06T16:34:39.411Z","trace_propagation_headers":{"sentry-trace":"db495cd8710643cd8912ae001f0b4e20-96e2b746be4b43ef","baggage":"sentry-trace_id=db495cd8710643cd8912ae001f0b4e20,sentry-release=Unknown"},"correlation_id":"01M1VS4MWYDTC9Q9D31N44V00V","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dc59d8dcc99b2ad20e06a3d92745079e9ef2965b67a570211185faae6c17e7c2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:39.413Z","job_size_bytes":2592,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-000244073b1a841032d71328: done: 0.01667 sec","job_status":"done","queue_duration_s":0.068227,"scheduling_latency_s":0.06824,"redis_calls":7,"redis_duration_s":0.001486,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01667,"completed_at":"2026-09-06T16:34:39.498Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1901gitlab # [ 251.569717] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.499Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"08b44862f1128c1840f9cf96","created_at":"2026-09-06T16:34:39.442Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-bcb0c19ddf4c499c","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:39.445Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-08b44862f1128c1840f9cf96: start","job_status":"start","queue_duration_s":0.053742,"scheduling_latency_s":0.053756}1902gitlab # [ 251.570370] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:39.565Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"08b44862f1128c1840f9cf96","created_at":"2026-09-06T16:34:39.442Z","trace_propagation_headers":{"sentry-trace":"91d22d3872f147938c0a72f579d6af6b-bcb0c19ddf4c499c","baggage":"sentry-trace_id=91d22d3872f147938c0a72f579d6af6b,sentry-release=Unknown"},"correlation_id":"01M1VS38T6FT6GG0W5MPFD4J36","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:39.445Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-08b44862f1128c1840f9cf96: done: 0.066244 sec","job_status":"done","queue_duration_s":0.053742,"scheduling_latency_s":0.053756,"redis_calls":11,"redis_duration_s":0.002489,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000356,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001655,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003872,"duration_s":0.066244,"completed_at":"2026-09-06T16:34:39.565Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1903gitlab # [ 251.787441] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1391568339 wd_nsec: 13915682021904gitlab # [ 251.649208] gitaly[917]: time="2026-09-06T16:34:39.759Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=1208 command.spawn_token_fork_ms=1204 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4SRSSVEYD84FA2DEKN4P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:38.548" grpc.time_ms=1210.416 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1905gitlab # [ 251.662290] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 12821906gitlab # [ 251.716661] gitaly[917]: time="2026-09-06T16:34:39.826Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V0XV866QZXVE033DP9T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:39.816" pid=917 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1907gitlab # [ 251.717844] gitaly[917]: time="2026-09-06T16:34:39.826Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V0XV866QZXVE033DP9T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:39.816" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1908gitlab # [ 251.718780] gitlab-workhorse[826]: time="2026-09-06T16:34:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671909gitlab # [ 251.719802] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 531910gitlab # [ 251.720571] gitaly[917]: time="2026-09-06T16:34:39.827Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V0XV866QZXVE033DP9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:39.816" grpc.time_ms=10.472 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1911gitlab # [ 251.794738] gitaly[917]: time="2026-09-06T16:34:39.904Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V2PJFKH09XRYB6PVSME grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:39.886" pid=917 remote_ip="2001:db8:1::1" request_sha=9777bf1e7c653bcaa4c50e8e5b128d10d36c9edd response_bytes=336 span.kind=server system=grpc1912gitlab # [ 251.795830] gitaly[917]: time="2026-09-06T16:34:39.905Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1201 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V2PJFKH09XRYB6PVSME grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:39.886" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1913gitlab # [ 251.796740] gitlab-workhorse[826]: time="2026-09-06T16:34:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361914gitlab # [ 251.797551] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 751915gitlab # [ 251.798259] gitaly[917]: time="2026-09-06T16:34:39.905Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1201 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V2PJFKH09XRYB6PVSME grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:39.886" grpc.time_ms=18.871 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1916gitlab # [ 251.865425] nixbot[2660]: {"ts": "2026-09-06T16:34:39.974Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1917gitlab # [ 251.934406] gitaly[917]: time="2026-09-06T16:34:40.043Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.042" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.042" grpc.time_ms=0.867 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 251.974437] gitaly[917]: time="2026-09-06T16:34:40.084Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V74Z2KFRBEDHR372SSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.077" grpc.time_ms=6.7 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1919gitlab # [ 251.975712] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1121920gitlab # [ 251.983943] gitaly[917]: time="2026-09-06T16:34:40.094Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:50.038" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.088" grpc.time_ms=5.53 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 252.010795] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.120Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7dc5fa443dc7380f0fb19ff1","created_at":"2026-09-06T16:34:40.115Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-cab8ece6beff4ea2","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.119Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7dc5fa443dc7380f0fb19ff1: start","job_status":"start","queue_duration_s":0.001356,"scheduling_latency_s":0.001389}1922gitlab # [ 252.021554] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1431923gitlab # [ 252.029412] nixbot[2660]: {"ts": "2026-09-06T16:34:40.139Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1924gitlab # [ 252.077763] gitaly[917]: time="2026-09-06T16:34:40.188Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VCE1X63FZVV6MRHD156 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.187" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.187" grpc.time_ms=0.412 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 252.089726] gitaly[917]: time="2026-09-06T16:34:40.199Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VCB77VT4C00QDKHKYGX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.190" pid=917 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1926gitlab # [ 252.090565] gitaly[917]: time="2026-09-06T16:34:40.200Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VCB77VT4C00QDKHKYGX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.190" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1927gitlab # [ 252.091158] gitlab-workhorse[826]: time="2026-09-06T16:34:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671928gitlab # [ 252.091553] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 611929gitlab # [ 252.093234] gitaly[917]: time="2026-09-06T16:34:40.200Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VCB77VT4C00QDKHKYGX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.190" grpc.time_ms=9.73 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1930gitlab # [ 252.130321] gitaly[917]: time="2026-09-06T16:34:40.240Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=21 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VCE1X63FZVV6MRHD156 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:50.163" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.213" grpc.time_ms=26.684 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931gitlab # [ 252.136811] gitaly[917]: time="2026-09-06T16:34:40.246Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.246" grpc.time_ms=0.309 pid=917 span.kind=server system=grpc1932gitlab # [ 252.172460] gitaly[917]: time="2026-09-06T16:34:40.282Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3889 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.282" grpc.time_ms=0.516 pid=917 span.kind=server system=grpc1933gitlab # [ 252.193409] gitaly[917]: time="2026-09-06T16:34:40.302Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.302" grpc.time_ms=0.324 pid=917 span.kind=server system=grpc1934gitlab # [ 252.196766] gitaly[917]: time="2026-09-06T16:34:40.306Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VG6W4J3T8R38QQ2W6XM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.298" grpc.time_ms=7.417 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1935gitlab # [ 252.197886] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 441936gitlab # [ 252.223363] gitaly[917]: time="2026-09-06T16:34:40.333Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 diskcache=de86f689-358f-47d8-affe-bc2e91bac0b1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.077" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.177" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937gitlab # [ 252.224325] gitaly[917]: time="2026-09-06T16:34:40.334Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=83 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7518 command.oublock=0 command.real_time_ms=148 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.077" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.177" grpc.time_ms=156.546 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938gitlab # [ 252.253123] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2201939gitlab # [ 252.254195] nixbot[2660]: {"ts": "2026-09-06T16:34:40.364Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1940gitlab # [ 252.271600] gitaly[917]: time="2026-09-06T16:34:40.381Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:10.280" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.381" grpc.time_ms=0.649 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1941gitlab # [ 252.272193] gitaly[917]: time="2026-09-06T16:34:40.382Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VHMDC8NVZXJNBYJNGJT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.369" pid=917 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1942gitlab # [ 252.272958] gitaly[917]: time="2026-09-06T16:34:40.382Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VHMDC8NVZXJNBYJNGJT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.369" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1943gitlab # [ 252.273694] gitlab-workhorse[826]: time="2026-09-06T16:34:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161944gitlab # [ 252.275234] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 741945gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'1946gitlab # [ 252.276454] gitaly[917]: time="2026-09-06T16:34:40.382Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VHMDC8NVZXJNBYJNGJT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:40.369" grpc.time_ms=13.37 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc1947gitlab # [ 252.296027] gitaly[917]: time="2026-09-06T16:34:40.406Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VKGDVHQ86YCBR522BNV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.105" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.405" grpc.time_ms=0.369 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1948gitlab # [ 252.336382] gitaly[917]: time="2026-09-06T16:34:40.446Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VKGDVHQ86YCBR522BNV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:50.040" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.440" grpc.time_ms=5.809 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1949gitlab # [ 252.356687] gitaly[917]: time="2026-09-06T16:34:40.466Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.365" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.466" grpc.time_ms=0.593 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1950gitlab # [ 252.363960] nixbot[2660]: INFO: ::ffff:127.0.0.1:60238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK1951gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.06 seconds)1952gitlab # [ 252.392688] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1331953gitlab # [ 252.393298] nixbot[2660]: {"ts": "2026-09-06T16:34:40.503Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e485f2cbba70e6f175cf8098620ece9de959d0ad \"HTTP/1.1 201 Created\""}1954gitlab # [ 252.402934] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.512Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"8ea8d6a660cb86d7962388ab","created_at":"2026-09-06T16:34:40.510Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"7dc5fa443dc7380f0fb19ff1","job_size_bytes":3,"pid":2544,"job_status":"deduplicated","message":"PipelineProcessWorker JID-8ea8d6a660cb86d7962388ab: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1955gitlab # [ 252.404315] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.514Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7dc5fa443dc7380f0fb19ff1","created_at":"2026-09-06T16:34:40.115Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-cab8ece6beff4ea2","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.119Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2544,"job_status":"rescheduled","message":"PipelineProcessWorker JID-7dc5fa443dc7380f0fb19ff1: rescheduled"}1956gitlab # [ 252.409683] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7dc5fa443dc7380f0fb19ff1","created_at":"2026-09-06T16:34:40.115Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-cab8ece6beff4ea2","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.119Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7dc5fa443dc7380f0fb19ff1: done: 0.398966 sec","job_status":"done","queue_duration_s":0.001356,"scheduling_latency_s":0.001389,"gitaly_calls":3,"gitaly_duration_s":0.223645,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.01942,"redis_read_bytes":226,"redis_write_bytes":18463,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001451,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000654,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003404,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12309,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.008079,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000208,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000703,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00347,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.117063,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.330459597,"duration_s":0.398966,"completed_at":"2026-09-06T16:34:40.519Z","load_balancing_strategy":"primary","db_duration_s":0.034599,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1957gitlab # [ 252.450587] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.521Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c68bbecdaa7034110c904bc4","created_at":"2026-09-06T16:34:40.171Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.173Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c68bbecdaa7034110c904bc4: start","job_status":"start","queue_duration_s":0.348329,"scheduling_latency_s":0.34835}1958gitlab # [ 252.451827] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.554Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c68bbecdaa7034110c904bc4","created_at":"2026-09-06T16:34:40.171Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.173Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c68bbecdaa7034110c904bc4: done: 0.032957 sec","job_status":"done","queue_duration_s":0.348329,"scheduling_latency_s":0.34835,"redis_calls":7,"redis_duration_s":0.001184,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032957,"completed_at":"2026-09-06T16:34:40.554Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007163,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1959gitlab # [ 252.507378] gitaly[917]: time="2026-09-06T16:34:40.531Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.530" grpc.time_ms=0.881 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 252.508208] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.555Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"70377e4e5483665a29fb9449","created_at":"2026-09-06T16:34:40.339Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.341Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-70377e4e5483665a29fb9449: start","job_status":"start","queue_duration_s":0.213842,"scheduling_latency_s":0.213863}1961gitlab # [ 252.520434] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.576Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"70377e4e5483665a29fb9449","created_at":"2026-09-06T16:34:40.339Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.341Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-70377e4e5483665a29fb9449: done: 0.020929 sec","job_status":"done","queue_duration_s":0.213842,"scheduling_latency_s":0.213863,"redis_calls":7,"redis_duration_s":0.003503,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020929,"completed_at":"2026-09-06T16:34:40.576Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 252.586684] gitaly[917]: time="2026-09-06T16:34:40.542Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:35:10.036" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.536" grpc.time_ms=5.485 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 252.587615] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.578Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"fb4fbf99d1f5326bba8f7d97","created_at":"2026-09-06T16:34:40.348Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.350Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-fb4fbf99d1f5326bba8f7d97: start","job_status":"start","queue_duration_s":0.227213,"scheduling_latency_s":0.227229}1964gitlab # [ 252.588641] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.600Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"fb4fbf99d1f5326bba8f7d97","created_at":"2026-09-06T16:34:40.348Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.350Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-fb4fbf99d1f5326bba8f7d97: done: 0.022258 sec","job_status":"done","queue_duration_s":0.227213,"scheduling_latency_s":0.227229,"redis_calls":7,"redis_duration_s":0.0017590000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022258,"completed_at":"2026-09-06T16:34:40.600Z","load_balancing_strategy":"primary","db_duration_s":0.003292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 252.628293] gitaly[917]: time="2026-09-06T16:34:40.552Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:50.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.546" grpc.time_ms=5.951 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 252.633654] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2391967gitlab # [ 252.636502] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.601Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f2a00f20d2b4a00439f206a7","created_at":"2026-09-06T16:34:40.357Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.358Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f2a00f20d2b4a00439f206a7: start","job_status":"start","queue_duration_s":0.243058,"scheduling_latency_s":0.243079}1968gitlab # [ 252.647973] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.623Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"f2a00f20d2b4a00439f206a7","created_at":"2026-09-06T16:34:40.357Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.358Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-f2a00f20d2b4a00439f206a7: done: 0.021623 sec","job_status":"done","queue_duration_s":0.243058,"scheduling_latency_s":0.243079,"redis_calls":10,"redis_duration_s":0.001349,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00036,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021623,"completed_at":"2026-09-06T16:34:40.623Z","load_balancing_strategy":"primary","db_duration_s":0.003182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969gitlab # [ 252.687388] gitaly[917]: time="2026-09-06T16:34:40.580Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:34:50.039" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:40.580" grpc.time_ms=0.194 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1970gitlab # [ 252.727842] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.624Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b6561fb265e6fc4016ab591b","created_at":"2026-09-06T16:34:40.358Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.360Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b6561fb265e6fc4016ab591b: start","job_status":"start","queue_duration_s":0.264418,"scheduling_latency_s":0.264433}1971gitlab # [ 252.728641] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.640Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b6561fb265e6fc4016ab591b","created_at":"2026-09-06T16:34:40.358Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.360Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b6561fb265e6fc4016ab591b: done: 0.016114 sec","job_status":"done","queue_duration_s":0.264418,"scheduling_latency_s":0.264433,"redis_calls":7,"redis_duration_s":0.001199,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016114,"completed_at":"2026-09-06T16:34:40.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1972gitlab # [ 252.772214] gitaly[917]: time="2026-09-06T16:34:40.591Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:34:50.045" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.585" grpc.time_ms=5.275 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1973gitlab # [ 252.772939] gitaly[917]: time="2026-09-06T16:34:40.632Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4147 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.632" grpc.time_ms=0.345 pid=917 span.kind=server system=grpc1974gitlab # [ 252.773642] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.641Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"78161facbf296171ad7a791f","created_at":"2026-09-06T16:34:40.360Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.361Z","job_size_bytes":19,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-78161facbf296171ad7a791f: start","job_status":"start","queue_duration_s":0.279755,"scheduling_latency_s":0.27977}1975gitlab # [ 252.825740] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.656Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"78161facbf296171ad7a791f","created_at":"2026-09-06T16:34:40.360Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.361Z","job_size_bytes":19,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-78161facbf296171ad7a791f: done: 0.01491 sec","job_status":"done","queue_duration_s":0.279755,"scheduling_latency_s":0.27977,"redis_calls":8,"redis_duration_s":0.001069,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01491,"completed_at":"2026-09-06T16:34:40.656Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1976gitlab # [ 252.826935] gitaly[917]: time="2026-09-06T16:34:40.652Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4149 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.652" grpc.time_ms=0.236 pid=917 span.kind=server system=grpc1977gitlab # [ 252.834451] gitaly[917]: time="2026-09-06T16:34:40.662Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M diskcache=cc82158e-4865-45f1-a7e7-413b3395c8ae grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:34:50.052" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:40.592" pid=917 ref=refs/keep-around/0984b3f14362b0077b7998aa4bc82cc88650f4d3 remote_ip="2001:db8:1::1" revision=0984b3f14362b0077b7998aa4bc82cc88650f4d3 span.kind=server system=grpc username=root1978gitlab # [ 252.844834] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.657Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fb2f923d6e04d10923a4a06e","created_at":"2026-09-06T16:34:40.362Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.366Z","job_size_bytes":66,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fb2f923d6e04d10923a4a06e: start","job_status":"start","queue_duration_s":0.291314,"scheduling_latency_s":0.291329}1979gitlab # [ 252.890306] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.672Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fb2f923d6e04d10923a4a06e","created_at":"2026-09-06T16:34:40.362Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.366Z","job_size_bytes":66,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fb2f923d6e04d10923a4a06e: done: 0.014936 sec","job_status":"done","queue_duration_s":0.291314,"scheduling_latency_s":0.291329,"redis_calls":7,"redis_duration_s":0.0020410000000000003,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014936,"completed_at":"2026-09-06T16:34:40.672Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 252.930094] gitaly[917]: time="2026-09-06T16:34:40.662Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=38 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5149 command.oublock=8 command.real_time_ms=61 command.spawn_token_fork_ms=1 command.system_time_ms=26 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:34:50.052" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:34:40.592" grpc.time_ms=69.327 pid=917 ref=refs/keep-around/0984b3f14362b0077b7998aa4bc82cc88650f4d3 remote_ip="2001:db8:1::1" revision=0984b3f14362b0077b7998aa4bc82cc88650f4d3 span.kind=server system=grpc username=root1981gitlab # [ 252.930762] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.673Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"412b1d613d3ee2e4fdcda77a","created_at":"2026-09-06T16:34:40.366Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.367Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-412b1d613d3ee2e4fdcda77a: start","job_status":"start","queue_duration_s":0.306001,"scheduling_latency_s":0.306015}1982gitlab # [ 252.931513] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.693Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"412b1d613d3ee2e4fdcda77a","created_at":"2026-09-06T16:34:40.366Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.367Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-412b1d613d3ee2e4fdcda77a: done: 0.019326 sec","job_status":"done","queue_duration_s":0.306001,"scheduling_latency_s":0.306015,"redis_calls":7,"redis_duration_s":0.0018089999999999998,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001403,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019326,"completed_at":"2026-09-06T16:34:40.692Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1983gitlab # [ 252.972223] gitaly[917]: time="2026-09-06T16:34:40.666Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.165" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.665" grpc.time_ms=0.496 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 253.014798] gitaly[917]: time="2026-09-06T16:34:40.811Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.811" grpc.time_ms=0.412 pid=917 span.kind=server system=grpc1985gitlab # [ 253.015692] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.694Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f27b9366085539eee0dca217","created_at":"2026-09-06T16:34:40.367Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.368Z","job_size_bytes":6,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f27b9366085539eee0dca217: start","job_status":"start","queue_duration_s":0.325701,"scheduling_latency_s":0.325726}1986gitlab # [ 253.016484] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.707Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"f27b9366085539eee0dca217","created_at":"2026-09-06T16:34:40.367Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.368Z","job_size_bytes":6,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-f27b9366085539eee0dca217: done: 0.012766 sec","job_status":"done","queue_duration_s":0.325701,"scheduling_latency_s":0.325726,"redis_calls":7,"redis_duration_s":0.002496,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012766,"completed_at":"2026-09-06T16:34:40.707Z","load_balancing_strategy":"primary","db_duration_s":0.000512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1987gitlab # [ 253.058748] gitaly[917]: time="2026-09-06T16:34:40.830Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3889 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.829" grpc.time_ms=0.804 pid=917 span.kind=server system=grpc1988gitlab # [ 253.059544] gitaly[917]: time="2026-09-06T16:34:40.851Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:40.850" grpc.time_ms=0.916 pid=917 span.kind=server system=grpc1989gitlab # [ 253.060210] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.708Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"cfd6240035ab58639382f02c","created_at":"2026-09-06T16:34:40.369Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.370Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cfd6240035ab58639382f02c: start","job_status":"start","queue_duration_s":0.337813,"scheduling_latency_s":0.337827}1990gitlab # [ 253.098787] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.724Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"cfd6240035ab58639382f02c","created_at":"2026-09-06T16:34:40.369Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.370Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cfd6240035ab58639382f02c: done: 0.015875 sec","job_status":"done","queue_duration_s":0.337813,"scheduling_latency_s":0.337827,"redis_calls":7,"redis_duration_s":0.001972,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015875,"completed_at":"2026-09-06T16:34:40.724Z","load_balancing_strategy":"primary","db_duration_s":0.00222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 253.105758] gitaly[917]: time="2026-09-06T16:34:40.854Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.687" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.787" operations="[repack_refs]" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1992gitlab # [ 253.114767] gitaly[917]: time="2026-09-06T16:34:40.856Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.687" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.787" pid=917 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-06 16:33:41.804410734 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root1993gitlab # [ 253.125842] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8f3a64f56ebf7aed7a2575fd","created_at":"2026-09-06T16:34:40.514Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.515Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8f3a64f56ebf7aed7a2575fd: start","job_status":"start","queue_duration_s":0.209402,"scheduling_latency_s":0.209439}1994gitlab # [ 253.140556] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.931Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8f3a64f56ebf7aed7a2575fd","created_at":"2026-09-06T16:34:40.514Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-22711771555e4cca","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.515Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8f3a64f56ebf7aed7a2575fd: done: 0.206125 sec","job_status":"done","queue_duration_s":0.209402,"scheduling_latency_s":0.209439,"gitaly_calls":3,"gitaly_duration_s":0.080861,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012762,"redis_read_bytes":144,"redis_write_bytes":16720,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001406,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001275,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.002096,"redis_queues_read_bytes":24,"redis_queues_write_bytes":11226,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.003161,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000975,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000886,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001557,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079298,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.182736354,"duration_s":0.206125,"completed_at":"2026-09-06T16:34:40.931Z","load_balancing_strategy":"primary","db_duration_s":0.038015,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1995gitlab # [ 253.185763] gitaly[917]: time="2026-09-06T16:34:40.861Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 diskcache=8982a4f2-af44-44ff-97f5-9f5d007be8f3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.687" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.787" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996gitlab # [ 253.228156] gitaly[917]: time="2026-09-06T16:34:40.861Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=70 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7620 command.oublock=0 command.real_time_ms=68 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.687" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.787" grpc.time_ms=73.658 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997gitlab # [ 253.228868] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.932Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0d58c75b47d1842274d0487d","created_at":"2026-09-06T16:34:40.668Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-e375d8af7d7f4a15","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.672Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0d58c75b47d1842274d0487d: start","job_status":"start","queue_duration_s":0.260723,"scheduling_latency_s":0.260738}1998gitlab # [ 253.229493] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.957Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0d58c75b47d1842274d0487d","created_at":"2026-09-06T16:34:40.668Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-e375d8af7d7f4a15","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.672Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0d58c75b47d1842274d0487d: done: 0.02451 sec","job_status":"done","queue_duration_s":0.260723,"scheduling_latency_s":0.260738,"redis_calls":7,"redis_duration_s":0.001108,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02451,"completed_at":"2026-09-06T16:34:40.957Z","load_balancing_strategy":"primary","db_duration_s":0.006763,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1999gitlab # [ 253.258670] gitaly[917]: time="2026-09-06T16:34:40.870Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.687" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.787" pid=917 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2000gitlab # [ 253.270502] gitaly[917]: time="2026-09-06T16:34:40.870Z" level=info msg="optimized repository" clean_stale_data_time=13.637208ms clean_work_trees_time="108.114µs" commit_graph_time="4.191µs" correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.749µs" pid=917 prune_time="3.353µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="5.308µs" storage_name=default system=housekeeping total_time=13.866847ms2001gitlab # [ 253.275957] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.958Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"5d3a819218bb0637343cbdff","created_at":"2026-09-06T16:34:40.672Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-e375d8af7d7f4a15","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.674Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-5d3a819218bb0637343cbdff: start","job_status":"start","queue_duration_s":0.283902,"scheduling_latency_s":0.283923}2002gitlab # [ 253.327242] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.993Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"5d3a819218bb0637343cbdff","created_at":"2026-09-06T16:34:40.672Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-e375d8af7d7f4a15","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.674Z","job_size_bytes":65,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-5d3a819218bb0637343cbdff: done: 0.035253 sec","job_status":"done","queue_duration_s":0.283902,"scheduling_latency_s":0.283923,"redis_calls":10,"redis_duration_s":0.001473,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035253,"completed_at":"2026-09-06T16:34:40.993Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3'2004gitlab # [ 253.330147] gitaly[917]: time="2026-09-06T16:34:40.870Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:10.687" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:40.787" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2005gitlab # [ 253.368885] gitaly[917]: time="2026-09-06T16:34:40.901Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:10.799" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.900" grpc.time_ms=0.851 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2006gitlab # [ 253.369586] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:40.994Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d30ce576fabe50f5875eb60c","created_at":"2026-09-06T16:34:40.711Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-e375d8af7d7f4a15","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.713Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d30ce576fabe50f5875eb60c: start","job_status":"start","queue_duration_s":0.281011,"scheduling_latency_s":0.281033}2007gitlab # [ 253.370192] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.072Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d30ce576fabe50f5875eb60c","created_at":"2026-09-06T16:34:40.711Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-e375d8af7d7f4a15","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.713Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d30ce576fabe50f5875eb60c: done: 0.077229 sec","job_status":"done","queue_duration_s":0.281011,"scheduling_latency_s":0.281033,"gitaly_calls":2,"gitaly_duration_s":0.004936,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004473,"redis_read_bytes":80,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000791,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000654,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000191,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000171,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00029,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000601,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054462,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.056902762,"duration_s":0.077229,"completed_at":"2026-09-06T16:34:41.072Z","load_balancing_strategy":"primary","db_duration_s":0.01674,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2008gitlab # [ 253.412730] gitaly[917]: time="2026-09-06T16:34:40.923Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4V7MT3V9416Q7YY9GY20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:10.822" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:40.923" grpc.time_ms=0.522 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2009gitlab # [ 253.452472] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.073Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ebc8ea0b2cf1b536c0dc8cf5","created_at":"2026-09-06T16:34:40.780Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.781Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ebc8ea0b2cf1b536c0dc8cf5: start","job_status":"start","queue_duration_s":0.291244,"scheduling_latency_s":0.291259}2010gitlab # [ 253.453214] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.097Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ebc8ea0b2cf1b536c0dc8cf5","created_at":"2026-09-06T16:34:40.780Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.781Z","job_size_bytes":23,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ebc8ea0b2cf1b536c0dc8cf5: done: 0.023883 sec","job_status":"done","queue_duration_s":0.291244,"scheduling_latency_s":0.291259,"redis_calls":7,"redis_duration_s":0.001004,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023883,"completed_at":"2026-09-06T16:34:41.097Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004226,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2011gitlab # [ 253.497574] gitaly[917]: time="2026-09-06T16:34:41.044Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:11.042" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:41.042" grpc.time_ms=0.963 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012gitlab # [ 253.498263] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.098Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"31723aef5ff3dcd612b9e777","created_at":"2026-09-06T16:34:40.866Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.868Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-31723aef5ff3dcd612b9e777: start","job_status":"start","queue_duration_s":0.229397,"scheduling_latency_s":0.229411}2013gitlab # [ 253.498847] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.114Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"31723aef5ff3dcd612b9e777","created_at":"2026-09-06T16:34:40.866Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.868Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-31723aef5ff3dcd612b9e777: done: 0.015929 sec","job_status":"done","queue_duration_s":0.229397,"scheduling_latency_s":0.229411,"redis_calls":7,"redis_duration_s":0.001032,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015929,"completed_at":"2026-09-06T16:34:41.114Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 253.540553] gitaly[917]: time="2026-09-06T16:34:41.065Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4VQ4ZMKWQ1JG2SJY7D9M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:11.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:41.064" grpc.time_ms=0.516 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015gitlab # [ 253.580624] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2a479f96f32778a47e2d038b","created_at":"2026-09-06T16:34:40.874Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.877Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2a479f96f32778a47e2d038b: start","job_status":"start","queue_duration_s":0.238149,"scheduling_latency_s":0.238164}2016gitlab # [ 253.581221] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.131Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2a479f96f32778a47e2d038b","created_at":"2026-09-06T16:34:40.874Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.877Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2a479f96f32778a47e2d038b: done: 0.016597 sec","job_status":"done","queue_duration_s":0.238149,"scheduling_latency_s":0.238164,"redis_calls":7,"redis_duration_s":0.0009759999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016597,"completed_at":"2026-09-06T16:34:41.131Z","load_balancing_strategy":"primary","db_duration_s":0.003283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 253.626300] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.132Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bda95e9df39f193757abafd0","created_at":"2026-09-06T16:34:40.883Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.884Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bda95e9df39f193757abafd0: start","job_status":"start","queue_duration_s":0.248233,"scheduling_latency_s":0.248249}2018gitlab # [ 253.626885] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.150Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bda95e9df39f193757abafd0","created_at":"2026-09-06T16:34:40.883Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.884Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bda95e9df39f193757abafd0: done: 0.017764 sec","job_status":"done","queue_duration_s":0.248233,"scheduling_latency_s":0.248249,"redis_calls":10,"redis_duration_s":0.001283,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000371,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017764,"completed_at":"2026-09-06T16:34:41.150Z","load_balancing_strategy":"primary","db_duration_s":0.002796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 253.666048] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.151Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"0794d734e87c8b6c9f937497","created_at":"2026-09-06T16:34:40.885Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.886Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-0794d734e87c8b6c9f937497: start","job_status":"start","queue_duration_s":0.265821,"scheduling_latency_s":0.265835}2020gitlab # [ 253.666778] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"0794d734e87c8b6c9f937497","created_at":"2026-09-06T16:34:40.885Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.886Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-0794d734e87c8b6c9f937497: done: 0.015127 sec","job_status":"done","queue_duration_s":0.265821,"scheduling_latency_s":0.265835,"redis_calls":7,"redis_duration_s":0.000995,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00031,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015127,"completed_at":"2026-09-06T16:34:41.166Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2021gitlab # [ 253.710696] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f10df3850f4a8a585d256a1e","created_at":"2026-09-06T16:34:40.886Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.888Z","job_size_bytes":19,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f10df3850f4a8a585d256a1e: start","job_status":"start","queue_duration_s":0.279825,"scheduling_latency_s":0.27984}2022gitlab # [ 253.713170] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f10df3850f4a8a585d256a1e","created_at":"2026-09-06T16:34:40.886Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.888Z","job_size_bytes":19,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f10df3850f4a8a585d256a1e: done: 0.015434 sec","job_status":"done","queue_duration_s":0.279825,"scheduling_latency_s":0.27984,"redis_calls":8,"redis_duration_s":0.001165,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015434,"completed_at":"2026-09-06T16:34:41.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002157,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2023gitlab # [ 253.762144] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.184Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4903a6fbd9a7d516eb6a6f66","created_at":"2026-09-06T16:34:40.888Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.889Z","job_size_bytes":66,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4903a6fbd9a7d516eb6a6f66: start","job_status":"start","queue_duration_s":0.294573,"scheduling_latency_s":0.294587}2024gitlab # [ 253.766053] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.197Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4903a6fbd9a7d516eb6a6f66","created_at":"2026-09-06T16:34:40.888Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.889Z","job_size_bytes":66,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4903a6fbd9a7d516eb6a6f66: done: 0.01347 sec","job_status":"done","queue_duration_s":0.294573,"scheduling_latency_s":0.294587,"redis_calls":7,"redis_duration_s":0.000979,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01347,"completed_at":"2026-09-06T16:34:41.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2025gitlab # [ 253.807342] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.198Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"537df09b2a16e3701c220a16","created_at":"2026-09-06T16:34:40.889Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.890Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-537df09b2a16e3701c220a16: start","job_status":"start","queue_duration_s":0.307731,"scheduling_latency_s":0.307746}2026gitlab # [ 253.818763] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.214Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"537df09b2a16e3701c220a16","created_at":"2026-09-06T16:34:40.889Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.890Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-537df09b2a16e3701c220a16: done: 0.015546 sec","job_status":"done","queue_duration_s":0.307731,"scheduling_latency_s":0.307746,"redis_calls":7,"redis_duration_s":0.001079,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015546,"completed_at":"2026-09-06T16:34:41.214Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2027gitlab # [ 253.877276] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a62f0d864778920e77685bad","created_at":"2026-09-06T16:34:40.891Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.891Z","job_size_bytes":6,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a62f0d864778920e77685bad: start","job_status":"start","queue_duration_s":0.323031,"scheduling_latency_s":0.323046}2028gitlab # [ 253.877405] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.223Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a62f0d864778920e77685bad","created_at":"2026-09-06T16:34:40.891Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.891Z","job_size_bytes":6,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a62f0d864778920e77685bad: done: 0.008358 sec","job_status":"done","queue_duration_s":0.323031,"scheduling_latency_s":0.323046,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008358,"completed_at":"2026-09-06T16:34:41.223Z","load_balancing_strategy":"primary","db_duration_s":0.000259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2029gitlab # [ 253.931600] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"da029c18c5577bb54bfdf0fd","created_at":"2026-09-06T16:34:40.892Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.893Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-da029c18c5577bb54bfdf0fd: start","job_status":"start","queue_duration_s":0.331146,"scheduling_latency_s":0.33116}2030gitlab # [ 253.932215] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"da029c18c5577bb54bfdf0fd","created_at":"2026-09-06T16:34:40.892Z","trace_propagation_headers":{"sentry-trace":"cb6386000cce438c91b0aed0af333788-56416732bb9042e0","baggage":"sentry-trace_id=cb6386000cce438c91b0aed0af333788,sentry-release=Unknown"},"correlation_id":"01M1VS4V7MT3V9416Q7YY9GY20","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:40.893Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-da029c18c5577bb54bfdf0fd: done: 0.013178 sec","job_status":"done","queue_duration_s":0.331146,"scheduling_latency_s":0.33116,"redis_calls":7,"redis_duration_s":0.000905,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013178,"completed_at":"2026-09-06T16:34:41.237Z","load_balancing_strategy":"primary","db_duration_s":0.001885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031gitlab # [ 253.982767] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"aaa485efecc52aaeddbc002f","created_at":"2026-09-06T16:34:41.032Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-30a695feb02f4446","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:41.034Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-aaa485efecc52aaeddbc002f: start","job_status":"start","queue_duration_s":0.204197,"scheduling_latency_s":0.204209}2032gitlab # [ 253.983397] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:41.257Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"aaa485efecc52aaeddbc002f","created_at":"2026-09-06T16:34:41.032Z","trace_propagation_headers":{"sentry-trace":"6cb3e706f383424ebd93e34a874158c7-30a695feb02f4446","baggage":"sentry-trace_id=6cb3e706f383424ebd93e34a874158c7,sentry-release=Unknown"},"correlation_id":"01M1VS4VQ4ZMKWQ1JG2SJY7D9M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:41.034Z","job_size_bytes":3,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-aaa485efecc52aaeddbc002f: done: 0.018961 sec","job_status":"done","queue_duration_s":0.204197,"scheduling_latency_s":0.204209,"redis_calls":7,"redis_duration_s":0.000961,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018961,"completed_at":"2026-09-06T16:34:41.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033gitlab # [ 254.051525] nixbot[2660]: {"ts": "2026-09-06T16:34:42.161Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 \"HTTP/1.1 201 Created\""}2034gitlab # [ 254.059145] nixbot[2660]: INFO: ::ffff:127.0.0.1:35122 - "GET /api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 200 OK2035gitlab # [ 254.060798] nixbot[2660]: {"ts": "2026-09-06T16:34:42.171Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-0984b3f14362-4"}2036gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=0984b3f14362b0077b7998aa4bc82cc88650f4d3', in 0.71 seconds)2037(finished: subtest: push delivers the webhook, in 50.34 seconds)2038subtest: eval and build post commit statuses2039gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2040gitlab # [ 254.110234] gitaly[917]: time="2026-09-06T16:34:42.220Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS4XC2H8A1WX0K82V0RFGA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:42.214" grpc.time_ms=5.632 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc2041gitlab # [ 254.111109] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 412042gitlab # [ 254.168499] gitaly[917]: time="2026-09-06T16:34:42.278Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=1 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4XD09B2FC430YAFXP0JA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:12.274" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:42.275" grpc.time_ms=3.162 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043gitlab # [ 254.188916] nixbot[2660]: {"ts": "2026-09-06T16:34:42.299Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-0984b3f14362-4", "argv0": "bwrap"}2044gitlab # [ 254.208601] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1102045gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.13 seconds)2046gitlab # [ 254.243769] gitaly[917]: time="2026-09-06T16:34:42.353Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4XDESC26E85CVMQSYF7J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:42.343" pid=917 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2047gitlab # [ 254.245092] gitaly[917]: time="2026-09-06T16:34:42.353Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4XDESC26E85CVMQSYF7J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:42.343" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc2048gitlab # [ 254.246494] gitlab-workhorse[826]: time="2026-09-06T16:34:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672049gitlab # [ 254.247890] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1312050gitlab # [ 254.249037] gitaly[917]: time="2026-09-06T16:34:42.353Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4XDESC26E85CVMQSYF7J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:42.343" grpc.time_ms=10.037 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc2051gitlab # [ 254.294363] nix-daemon[2834]: accepted connection from pid 3667, user nixbot2052gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2053gitlab # [ 255.311759] gitaly[917]: time="2026-09-06T16:34:43.421Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4YGXDGE0N0NWF15BSHTT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:13.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:43.419" grpc.time_ms=0.951 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2054gitlab # [ 255.362418] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1132055gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.15 seconds)2056gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2057gitlab # [ 256.468679] gitaly[917]: time="2026-09-06T16:34:44.578Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS4ZN08C73V7A16CXYFHT7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:14.276" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:44.577" grpc.time_ms=0.998 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab # [ 256.525459] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1212059gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.16 seconds)2060gitlab # [ 256.996727] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.105Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"61697854122f938232b5590e","created_at":"2026-09-06T16:34:45.087Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d808d0540a8065d87c1cff76bc77fdb","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.102Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-61697854122f938232b5590e: start","job_status":"start","queue_duration_s":0.003118,"scheduling_latency_s":0.003178}2061gitlab # [ 257.084976] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.194Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"61697854122f938232b5590e","created_at":"2026-09-06T16:34:45.087Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d808d0540a8065d87c1cff76bc77fdb","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.102Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-61697854122f938232b5590e: done: 0.089483 sec","job_status":"done","queue_duration_s":0.003118,"scheduling_latency_s":0.003178,"redis_calls":12,"redis_duration_s":0.015066,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.004432,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000935,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001168864,"duration_s":0.089483,"completed_at":"2026-09-06T16:34:45.194Z","load_balancing_strategy":"primary","db_duration_s":0.000536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2062gitlab # [ 257.160753] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ff44405942031f7b294c4fef","created_at":"2026-09-06T16:34:45.141Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b66b8040228f95415cdcbf84f70fb585","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.150Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ff44405942031f7b294c4fef: start","job_status":"start","queue_duration_s":0.120157,"scheduling_latency_s":0.120211}2063gitlab # [ 257.219680] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.329Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2064gitlab # [ 257.242204] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ff44405942031f7b294c4fef","created_at":"2026-09-06T16:34:45.141Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b66b8040228f95415cdcbf84f70fb585","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.150Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ff44405942031f7b294c4fef: done: 0.081504 sec","job_status":"done","queue_duration_s":0.120157,"scheduling_latency_s":0.120211,"redis_calls":9,"redis_duration_s":0.030201,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.021897,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.081504,"completed_at":"2026-09-06T16:34:45.351Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 257.310568] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3248b6208dd9085fbb57a9b1","created_at":"2026-09-06T16:34:45.162Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0bfd8c88b57855d37536ff1cdcb0aae","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.175Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3248b6208dd9085fbb57a9b1: start","job_status":"start","queue_duration_s":0.244464,"scheduling_latency_s":0.244518}2066gitlab # [ 257.370598] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3248b6208dd9085fbb57a9b1","created_at":"2026-09-06T16:34:45.162Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0bfd8c88b57855d37536ff1cdcb0aae","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.175Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3248b6208dd9085fbb57a9b1: done: 0.060562 sec","job_status":"done","queue_duration_s":0.244464,"scheduling_latency_s":0.244518,"redis_calls":12,"redis_duration_s":0.013059,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002007,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005082,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00597,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012611938,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.060562,"completed_at":"2026-09-06T16:34:45.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011569,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2067gitlab # [ 257.428546] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f52d9b0b7324c5b386bfdf7e","created_at":"2026-09-06T16:34:45.185Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec62445168c7cd933de77091df5b792d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.269Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f52d9b0b7324c5b386bfdf7e: start","job_status":"start","queue_duration_s":0.269138,"scheduling_latency_s":0.269193}2068gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2069gitlab # [ 257.520089] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f52d9b0b7324c5b386bfdf7e","created_at":"2026-09-06T16:34:45.185Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec62445168c7cd933de77091df5b792d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.269Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f52d9b0b7324c5b386bfdf7e: done: 0.091609 sec","job_status":"done","queue_duration_s":0.269138,"scheduling_latency_s":0.269193,"redis_calls":9,"redis_duration_s":0.01308,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002731,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00911,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091609,"completed_at":"2026-09-06T16:34:45.629Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 257.592665] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3ee9244e342a0dd06efafb93","created_at":"2026-09-06T16:34:45.280Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20d9b8620103b7768f9056a5695ea1e0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.299Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3ee9244e342a0dd06efafb93: start","job_status":"start","queue_duration_s":0.403189,"scheduling_latency_s":0.403221}2071gitlab # [ 257.639368] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3ee9244e342a0dd06efafb93","created_at":"2026-09-06T16:34:45.280Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20d9b8620103b7768f9056a5695ea1e0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.299Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3ee9244e342a0dd06efafb93: done: 0.046945 sec","job_status":"done","queue_duration_s":0.403189,"scheduling_latency_s":0.403221,"redis_calls":11,"redis_duration_s":0.008660000000000001,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002784,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0035,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002376,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046945,"completed_at":"2026-09-06T16:34:45.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab # [ 257.661152] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"845c3dfa999ef7ce3591e6e4","created_at":"2026-09-06T16:34:45.330Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4bf4a004d2e13d60bc5c6b65367cdfb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.348Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-845c3dfa999ef7ce3591e6e4: start","job_status":"start","queue_duration_s":0.405477,"scheduling_latency_s":0.405505}2073gitlab # [ 257.663335] gitaly[917]: time="2026-09-06T16:34:45.770Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS50V014EZ8QMS89B40PEQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:15.462" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:45.763" grpc.time_ms=7.275 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2074gitlab # [ 257.664296] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.774Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2075gitlab # [ 257.682489] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 612076gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.21 seconds)2077gitlab # [ 257.683408] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"845c3dfa999ef7ce3591e6e4","created_at":"2026-09-06T16:34:45.330Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4bf4a004d2e13d60bc5c6b65367cdfb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.348Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-845c3dfa999ef7ce3591e6e4: done: 0.036903 sec","job_status":"done","queue_duration_s":0.405477,"scheduling_latency_s":0.405505,"redis_calls":9,"redis_duration_s":0.007817000000000001,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001186,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002624,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004007,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036903,"completed_at":"2026-09-06T16:34:45.791Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 257.706991] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"73e9141d2d09ceba347b728c","created_at":"2026-09-06T16:34:45.419Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53450fa7c8a384864759dee4ed6974d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:34:45.445Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-73e9141d2d09ceba347b728c: start","job_status":"start","queue_duration_s":0.353433,"scheduling_latency_s":0.353452}2079gitlab # [ 257.782957] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.893Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2080gitlab # [ 257.791049] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"73e9141d2d09ceba347b728c","created_at":"2026-09-06T16:34:45.419Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53450fa7c8a384864759dee4ed6974d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:34:45.445Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-73e9141d2d09ceba347b728c: done: 0.101789 sec","job_status":"done","queue_duration_s":0.353433,"scheduling_latency_s":0.353452,"redis_calls":9,"redis_duration_s":0.006798,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003343,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.08,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.08,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101789,"completed_at":"2026-09-06T16:34:45.900Z","load_balancing_strategy":"primary","db_duration_s":0.079631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 257.831486] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:45.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"89b1ddce442f3bd2241351bd","created_at":"2026-09-06T16:34:45.461Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"825e79ff0b7f21730894e227744ba6e3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.470Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-89b1ddce442f3bd2241351bd: start","job_status":"start","queue_duration_s":0.469552,"scheduling_latency_s":0.46961}2082gitlab # [ 257.904465] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.013Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"89b1ddce442f3bd2241351bd","created_at":"2026-09-06T16:34:45.461Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"825e79ff0b7f21730894e227744ba6e3","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.470Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-89b1ddce442f3bd2241351bd: done: 0.073585 sec","job_status":"done","queue_duration_s":0.469552,"scheduling_latency_s":0.46961,"redis_calls":10,"redis_duration_s":0.007163,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001102,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001452978,"duration_s":0.073585,"completed_at":"2026-09-06T16:34:46.013Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2083gitlab # [ 257.957869] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.067Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"272ed5836835087515db5aab","created_at":"2026-09-06T16:34:45.481Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"354e970619c209a4c353ebe6a8c578f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.540Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-272ed5836835087515db5aab: start","job_status":"start","queue_duration_s":0.526687,"scheduling_latency_s":0.526733}2084gitlab # [ 258.055211] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"272ed5836835087515db5aab","created_at":"2026-09-06T16:34:45.481Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"354e970619c209a4c353ebe6a8c578f9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.540Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-272ed5836835087515db5aab: done: 0.096913 sec","job_status":"done","queue_duration_s":0.526687,"scheduling_latency_s":0.526733,"redis_calls":11,"redis_duration_s":0.00355,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000855136,"exclusive_lock_hold_duration_s":0.040292272,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.096913,"completed_at":"2026-09-06T16:34:46.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 258.127831] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fdd70559dd537c555fad8742","created_at":"2026-09-06T16:34:45.569Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95980b545acac54638e30b252ca8d135","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.587Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fdd70559dd537c555fad8742: start","job_status":"start","queue_duration_s":0.650491,"scheduling_latency_s":0.65053}2086gitlab # [ 258.150170] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.260Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fdd70559dd537c555fad8742","created_at":"2026-09-06T16:34:45.569Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95980b545acac54638e30b252ca8d135","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.587Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fdd70559dd537c555fad8742: done: 0.02253 sec","job_status":"done","queue_duration_s":0.650491,"scheduling_latency_s":0.65053,"redis_calls":9,"redis_duration_s":0.002116,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02253,"completed_at":"2026-09-06T16:34:46.260Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 258.151917] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f2670de191283cf4ceec6b04","created_at":"2026-09-06T16:34:45.618Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75a63bc8945cbaedd98a8f9ce302355d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:34:45.700Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f2670de191283cf4ceec6b04: start","job_status":"start","queue_duration_s":0.560983,"scheduling_latency_s":0.560998}2088gitlab # [ 258.160938] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.269Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2089gitlab # [ 258.201936] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f2670de191283cf4ceec6b04","created_at":"2026-09-06T16:34:45.618Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"75a63bc8945cbaedd98a8f9ce302355d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:34:45.700Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f2670de191283cf4ceec6b04: done: 0.012687 sec","job_status":"done","queue_duration_s":0.560983,"scheduling_latency_s":0.560998,"redis_calls":9,"redis_duration_s":0.001875,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012687,"completed_at":"2026-09-06T16:34:46.273Z","load_balancing_strategy":"primary","db_duration_s":0.00073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab # [ 258.203567] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1ea6535047ed43975abe9ddd","created_at":"2026-09-06T16:34:45.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.714Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1ea6535047ed43975abe9ddd: start","job_status":"start","queue_duration_s":0.560948,"scheduling_latency_s":0.560963}2091gitlab # [ 258.244554] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1ea6535047ed43975abe9ddd","created_at":"2026-09-06T16:34:45.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.714Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1ea6535047ed43975abe9ddd: done: 0.030556 sec","job_status":"done","queue_duration_s":0.560948,"scheduling_latency_s":0.560963,"redis_calls":18,"redis_duration_s":0.004162,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000907,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000289,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002115,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000851,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030556,"completed_at":"2026-09-06T16:34:46.305Z","load_balancing_strategy":"primary","db_duration_s":0.002416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092gitlab # [ 258.285962] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"adf367ed1ca6ce12edc79aef","created_at":"2026-09-06T16:34:45.719Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e2cb7ec798a7201779c99acaa759cc7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.729Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-adf367ed1ca6ce12edc79aef: start","job_status":"start","queue_duration_s":0.579137,"scheduling_latency_s":0.579178}2093gitlab # [ 258.286799] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.325Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2094gitlab # [ 258.287673] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"adf367ed1ca6ce12edc79aef","created_at":"2026-09-06T16:34:45.719Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e2cb7ec798a7201779c99acaa759cc7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.729Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-adf367ed1ca6ce12edc79aef: done: 0.028581 sec","job_status":"done","queue_duration_s":0.579137,"scheduling_latency_s":0.579178,"redis_calls":9,"redis_duration_s":0.002063,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028581,"completed_at":"2026-09-06T16:34:46.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0008,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095gitlab # [ 258.306732] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7eb61b39320a0e7f3d4eee19","created_at":"2026-09-06T16:34:45.750Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aaa6e23cab4dae2e7cf2e02a50150e05","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.760Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7eb61b39320a0e7f3d4eee19: start","job_status":"start","queue_duration_s":0.578547,"scheduling_latency_s":0.578574}2096gitlab # [ 258.353886] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.350Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2097gitlab # [ 258.354499] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7eb61b39320a0e7f3d4eee19","created_at":"2026-09-06T16:34:45.750Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aaa6e23cab4dae2e7cf2e02a50150e05","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.760Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7eb61b39320a0e7f3d4eee19: done: 0.018913 sec","job_status":"done","queue_duration_s":0.578547,"scheduling_latency_s":0.578574,"redis_calls":9,"redis_duration_s":0.0017360000000000001,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018913,"completed_at":"2026-09-06T16:34:46.357Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098gitlab # [ 258.361668] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"f2f131e8e994172b04f8dd02","created_at":"2026-09-06T16:34:45.770Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1656f6cfd274c008015f72a4194d6c5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.781Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-f2f131e8e994172b04f8dd02: start","job_status":"start","queue_duration_s":0.577045,"scheduling_latency_s":0.577059}2099gitlab # [ 258.403754] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.374Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"f2f131e8e994172b04f8dd02","created_at":"2026-09-06T16:34:45.770Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1656f6cfd274c008015f72a4194d6c5","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.781Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-f2f131e8e994172b04f8dd02: done: 0.015891 sec","job_status":"done","queue_duration_s":0.577045,"scheduling_latency_s":0.577059,"redis_calls":12,"redis_duration_s":0.001665,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000585,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00033859,"exclusive_lock_hold_duration_s":0.004481296,"duration_s":0.015891,"completed_at":"2026-09-06T16:34:46.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100gitlab # [ 258.413124] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c4444b2332752e9f3759eda9","created_at":"2026-09-06T16:34:45.796Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37665d998debce72d16b497085040599","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.803Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c4444b2332752e9f3759eda9: start","job_status":"start","queue_duration_s":0.571202,"scheduling_latency_s":0.571214}2101gitlab # [ 258.439725] nix-daemon[2834]: accepted connection from pid 3716, user nixbot2102gitlab # [ 258.474664] nix-daemon[2834]: accepted connection from pid 3667, user nixbot2103gitlab # [ 258.526180] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c4444b2332752e9f3759eda9","created_at":"2026-09-06T16:34:45.796Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37665d998debce72d16b497085040599","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.803Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c4444b2332752e9f3759eda9: done: 0.260972 sec","job_status":"done","queue_duration_s":0.571202,"scheduling_latency_s":0.571214,"redis_calls":564,"redis_duration_s":0.083663,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.082666,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.197746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.260972,"completed_at":"2026-09-06T16:34:46.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104gitlab # [ 258.564429] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f78cbff5f7e505c605268ac0","created_at":"2026-09-06T16:34:45.805Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84cc5f88f6b7df1516483c5d5d19d241","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.810Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f78cbff5f7e505c605268ac0: start","job_status":"start","queue_duration_s":0.863708,"scheduling_latency_s":0.863737}2105gitlab # [ 258.568371] gitaly[917]: time="2026-09-06T16:34:46.678Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:16.377" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:46.677" grpc.time_ms=0.602 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106gitlab # [ 258.580595] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.690Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f78cbff5f7e505c605268ac0","created_at":"2026-09-06T16:34:45.805Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84cc5f88f6b7df1516483c5d5d19d241","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.810Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f78cbff5f7e505c605268ac0: done: 0.016305 sec","job_status":"done","queue_duration_s":0.863708,"scheduling_latency_s":0.863737,"redis_calls":11,"redis_duration_s":0.00183,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000379936,"exclusive_lock_hold_duration_s":0.001968406,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.016305,"completed_at":"2026-09-06T16:34:46.690Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107gitlab # [ 258.587830] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6bba9d696eb137e8da7167ae","created_at":"2026-09-06T16:34:45.812Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.815Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6bba9d696eb137e8da7167ae: start","job_status":"start","queue_duration_s":0.878414,"scheduling_latency_s":0.878429}2108gitlab # [ 258.648448] gitaly[917]: time="2026-09-06T16:34:46.758Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=52 command.spawn_token_fork_ms=46 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:56.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:46.704" grpc.time_ms=53.863 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2109gitlab # [ 258.649722] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6bba9d696eb137e8da7167ae","created_at":"2026-09-06T16:34:45.812Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.815Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6bba9d696eb137e8da7167ae: done: 0.064425 sec","job_status":"done","queue_duration_s":0.878414,"scheduling_latency_s":0.878429,"redis_calls":70,"redis_duration_s":0.010453,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001681,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000475,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.007568,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.064425,"completed_at":"2026-09-06T16:34:46.758Z","load_balancing_strategy":"primary","db_duration_s":0.00291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 258.659823] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7ff284782bc96419acf9a103","created_at":"2026-09-06T16:34:45.819Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d03bb969fdde6672c5e1e06447e7a501","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.822Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7ff284782bc96419acf9a103: start","job_status":"start","queue_duration_s":0.937021,"scheduling_latency_s":0.937042}2111gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2112gitlab # [ 258.702826] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.778Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7ff284782bc96419acf9a103","created_at":"2026-09-06T16:34:45.819Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d03bb969fdde6672c5e1e06447e7a501","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.822Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7ff284782bc96419acf9a103: done: 0.019288 sec","job_status":"done","queue_duration_s":0.937021,"scheduling_latency_s":0.937042,"redis_calls":9,"redis_duration_s":0.001329,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019288,"completed_at":"2026-09-06T16:34:46.778Z","load_balancing_strategy":"primary","db_duration_s":0.005486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113gitlab # [ 258.746816] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.779Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"14d86d867649bfcbfcb7d45a","created_at":"2026-09-06T16:34:45.888Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"782ac0a55add02695b729e773a48808a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.902Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-14d86d867649bfcbfcb7d45a: start","job_status":"start","queue_duration_s":0.877761,"scheduling_latency_s":0.877777}2114gitlab # [ 258.747677] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"14d86d867649bfcbfcb7d45a","created_at":"2026-09-06T16:34:45.888Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"782ac0a55add02695b729e773a48808a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.902Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-14d86d867649bfcbfcb7d45a: done: 0.014861 sec","job_status":"done","queue_duration_s":0.877761,"scheduling_latency_s":0.877777,"redis_calls":11,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000309257,"exclusive_lock_hold_duration_s":0.003574197,"duration_s":0.014861,"completed_at":"2026-09-06T16:34:46.794Z","load_balancing_strategy":"primary","db_duration_s":0.001631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115gitlab # [ 258.758521] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"48d111dc277f5ee06c05e209","created_at":"2026-09-06T16:34:45.944Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4e959b9168bc7cfafb5b0e98b9f5cfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.957Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-48d111dc277f5ee06c05e209: start","job_status":"start","queue_duration_s":0.838,"scheduling_latency_s":0.838014}2116gitlab # [ 258.844912] gitaly[917]: time="2026-09-06T16:34:46.954Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS51ZBT8TT8E4FXSRVK7JZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:16.452" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:46.953" grpc.time_ms=0.774 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2117gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.12 seconds)2118gitlab # [ 258.845671] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:46.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"48d111dc277f5ee06c05e209","created_at":"2026-09-06T16:34:45.944Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4e959b9168bc7cfafb5b0e98b9f5cfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.957Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-48d111dc277f5ee06c05e209: done: 0.158781 sec","job_status":"done","queue_duration_s":0.838,"scheduling_latency_s":0.838014,"redis_calls":13,"redis_duration_s":0.0019390000000000002,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000515,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000315123,"exclusive_lock_hold_duration_s":0.143949021,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.158781,"completed_at":"2026-09-06T16:34:46.954Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab # [ 258.860060] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 782120gitlab # [ 258.899493] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3a4ea069b5b4dfdf5939ef20","created_at":"2026-09-06T16:34:45.968Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dcc30539184082afae417f7413c07f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.977Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3a4ea069b5b4dfdf5939ef20: start","job_status":"start","queue_duration_s":1.031484,"scheduling_latency_s":1.031515}2121gitlab # [ 258.907811] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 4082122gitlab # [ 258.909076] nixbot[2660]: {"ts": "2026-09-06T16:34:47.018Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 \"HTTP/1.1 201 Created\""}2123gitlab # [ 258.946228] gitaly[917]: time="2026-09-06T16:34:47.056Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS523DH0RPKKKBMK03B85E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:17.055" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:47.055" grpc.time_ms=0.633 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2124gitlab # [ 258.953797] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3a4ea069b5b4dfdf5939ef20","created_at":"2026-09-06T16:34:45.968Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dcc30539184082afae417f7413c07f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.977Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3a4ea069b5b4dfdf5939ef20: done: 0.054346 sec","job_status":"done","queue_duration_s":1.031484,"scheduling_latency_s":1.031515,"redis_calls":13,"redis_duration_s":0.002486,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000788,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000726908,"exclusive_lock_hold_duration_s":0.014156827,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.054346,"completed_at":"2026-09-06T16:34:47.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 258.957683] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"cfbba0ac37604070f1a0a3b3","created_at":"2026-09-06T16:34:45.981Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2400cb4c280c8801cc42a1dffac11e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.990Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-cfbba0ac37604070f1a0a3b3: start","job_status":"start","queue_duration_s":1.074896,"scheduling_latency_s":1.074911}2126gitlab # [ 258.970844] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"cfbba0ac37604070f1a0a3b3","created_at":"2026-09-06T16:34:45.981Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2400cb4c280c8801cc42a1dffac11e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:34:45.990Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-cfbba0ac37604070f1a0a3b3: done: 0.014426 sec","job_status":"done","queue_duration_s":1.074896,"scheduling_latency_s":1.074911,"redis_calls":9,"redis_duration_s":0.001691,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000355,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014426,"completed_at":"2026-09-06T16:34:47.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2127gitlab # [ 259.017856] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.080Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"36876055a872520f305032f5","created_at":"2026-09-06T16:34:45.997Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e36a34fb061708ba2c079766604387e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:34:46.005Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-36876055a872520f305032f5: start","job_status":"start","queue_duration_s":1.075187,"scheduling_latency_s":1.075201}2128gitlab # [ 259.020195] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.102Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"36876055a872520f305032f5","created_at":"2026-09-06T16:34:45.997Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-d52fa8628a854af2","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e36a34fb061708ba2c079766604387e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:34:46.005Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-36876055a872520f305032f5: done: 0.022441 sec","job_status":"done","queue_duration_s":1.075187,"scheduling_latency_s":1.075201,"redis_calls":9,"redis_duration_s":0.001784,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022441,"completed_at":"2026-09-06T16:34:47.102Z","load_balancing_strategy":"primary","db_duration_s":0.00416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 259.066644] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.104Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9df581e91452ff46e4510ffc","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9df581e91452ff46e4510ffc: start","job_status":"start","queue_duration_s":0.810176,"scheduling_latency_s":0.810193}2130gitlab # [ 259.067665] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.116Z","message":"No existing merge request to be cleaned up."}2131gitlab # [ 259.068863] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.125Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9df581e91452ff46e4510ffc","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9df581e91452ff46e4510ffc: done: 0.0215 sec","job_status":"done","queue_duration_s":0.810176,"scheduling_latency_s":0.810193,"redis_calls":12,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0215,"completed_at":"2026-09-06T16:34:47.125Z","load_balancing_strategy":"primary","db_duration_s":0.003562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2132gitlab # [ 259.109916] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.126Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce2391ea3cf9e72ed2247f52","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce2391ea3cf9e72ed2247f52: start","job_status":"start","queue_duration_s":0.832614,"scheduling_latency_s":0.832629}2133gitlab # [ 259.115687] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.134Z","message":"No existing merge request to be cleaned up."}2134gitlab # [ 259.117701] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.140Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce2391ea3cf9e72ed2247f52","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce2391ea3cf9e72ed2247f52: done: 0.01349 sec","job_status":"done","queue_duration_s":0.832614,"scheduling_latency_s":0.832629,"redis_calls":10,"redis_duration_s":0.0024419999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01349,"completed_at":"2026-09-06T16:34:47.140Z","load_balancing_strategy":"primary","db_duration_s":0.001777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135gitlab # [ 259.164845] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.141Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"61fae184bccdc37fce216145","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-61fae184bccdc37fce216145: start","job_status":"start","queue_duration_s":0.84719,"scheduling_latency_s":0.847204}2136gitlab # [ 259.165654] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.148Z","message":"No existing merge request to be cleaned up."}2137gitlab # [ 259.166372] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.153Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"61fae184bccdc37fce216145","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-61fae184bccdc37fce216145: done: 0.012348 sec","job_status":"done","queue_duration_s":0.84719,"scheduling_latency_s":0.847204,"redis_calls":10,"redis_duration_s":0.002206,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012348,"completed_at":"2026-09-06T16:34:47.153Z","load_balancing_strategy":"primary","db_duration_s":0.001692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138gitlab # [ 259.208397] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.154Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"51021184d73c8ff23a0cd6cb","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-51021184d73c8ff23a0cd6cb: start","job_status":"start","queue_duration_s":0.860601,"scheduling_latency_s":0.860615}2139gitlab # [ 259.210790] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.161Z","message":"No existing merge request to be cleaned up."}2140gitlab # [ 259.212613] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.166Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"51021184d73c8ff23a0cd6cb","created_at":"2026-09-06T16:34:46.286Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-f2888d3e3f1f4796","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d4fe7611ed7a90ec5395bc493ea8eaac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.294Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-51021184d73c8ff23a0cd6cb: done: 0.012028 sec","job_status":"done","queue_duration_s":0.860601,"scheduling_latency_s":0.860615,"redis_calls":10,"redis_duration_s":0.002189,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012028,"completed_at":"2026-09-06T16:34:47.166Z","load_balancing_strategy":"primary","db_duration_s":0.00169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 259.262618] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef0ca8e05a62fcef23ee0647","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef0ca8e05a62fcef23ee0647: start","job_status":"start","queue_duration_s":0.415486,"scheduling_latency_s":0.4155}2142gitlab # [ 259.263518] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef0ca8e05a62fcef23ee0647","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef0ca8e05a62fcef23ee0647: done: 0.014176 sec","job_status":"done","queue_duration_s":0.415486,"scheduling_latency_s":0.4155,"redis_calls":15,"redis_duration_s":0.003011,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001785,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014176,"completed_at":"2026-09-06T16:34:47.181Z","load_balancing_strategy":"primary","db_duration_s":0.000468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 259.309221] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cef4d2001368f75c059a8f8a","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cef4d2001368f75c059a8f8a: start","job_status":"start","queue_duration_s":0.430764,"scheduling_latency_s":0.430787}2144gitlab # [ 259.309899] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cef4d2001368f75c059a8f8a","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cef4d2001368f75c059a8f8a: done: 0.010229 sec","job_status":"done","queue_duration_s":0.430764,"scheduling_latency_s":0.430787,"redis_calls":13,"redis_duration_s":0.002577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010229,"completed_at":"2026-09-06T16:34:47.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 259.353178] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18f29ebacfcf8b3c087c966b","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18f29ebacfcf8b3c087c966b: start","job_status":"start","queue_duration_s":0.442119,"scheduling_latency_s":0.442132}2146gitlab # [ 259.353891] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18f29ebacfcf8b3c087c966b","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18f29ebacfcf8b3c087c966b: done: 0.018398 sec","job_status":"done","queue_duration_s":0.442119,"scheduling_latency_s":0.442132,"redis_calls":13,"redis_duration_s":0.003673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002556,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018398,"completed_at":"2026-09-06T16:34:47.212Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147gitlab # [ 259.401138] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50fdbe1357ab0542dc8b6b3d","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50fdbe1357ab0542dc8b6b3d: start","job_status":"start","queue_duration_s":0.462596,"scheduling_latency_s":0.462634}2148gitlab # [ 259.401969] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50fdbe1357ab0542dc8b6b3d","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50fdbe1357ab0542dc8b6b3d: done: 0.018354 sec","job_status":"done","queue_duration_s":0.462596,"scheduling_latency_s":0.462634,"redis_calls":13,"redis_duration_s":0.00227,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001496,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018354,"completed_at":"2026-09-06T16:34:47.233Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149gitlab # [ 259.459685] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42f4a99dbfa606bccccd8230","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42f4a99dbfa606bccccd8230: start","job_status":"start","queue_duration_s":0.481763,"scheduling_latency_s":0.481787}2150gitlab # [ 259.460643] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42f4a99dbfa606bccccd8230","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42f4a99dbfa606bccccd8230: done: 0.009118 sec","job_status":"done","queue_duration_s":0.481763,"scheduling_latency_s":0.481787,"redis_calls":13,"redis_duration_s":0.001479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009118,"completed_at":"2026-09-06T16:34:47.243Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 259.481109] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e89cc223c474939f5fdd1b92","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e89cc223c474939f5fdd1b92: start","job_status":"start","queue_duration_s":0.491712,"scheduling_latency_s":0.491726}2152gitlab # [ 259.492657] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e89cc223c474939f5fdd1b92","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e89cc223c474939f5fdd1b92: done: 0.009081 sec","job_status":"done","queue_duration_s":0.491712,"scheduling_latency_s":0.491726,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009081,"completed_at":"2026-09-06T16:34:47.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2153gitlab # [ 259.534855] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daf5769ac13ad2f7be7f62c2","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daf5769ac13ad2f7be7f62c2: start","job_status":"start","queue_duration_s":0.501637,"scheduling_latency_s":0.501651}2154gitlab # [ 259.574051] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daf5769ac13ad2f7be7f62c2","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daf5769ac13ad2f7be7f62c2: done: 0.00922 sec","job_status":"done","queue_duration_s":0.501637,"scheduling_latency_s":0.501651,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00922,"completed_at":"2026-09-06T16:34:47.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab # [ 259.575741] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0c71e3ec19426b4e3f1aafb","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0c71e3ec19426b4e3f1aafb: start","job_status":"start","queue_duration_s":0.511695,"scheduling_latency_s":0.511709}2156gitlab # [ 259.585946] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0c71e3ec19426b4e3f1aafb","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0c71e3ec19426b4e3f1aafb: done: 0.009053 sec","job_status":"done","queue_duration_s":0.511695,"scheduling_latency_s":0.511709,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009053,"completed_at":"2026-09-06T16:34:47.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2157gitlab # [ 259.637104] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca4d30784529b8868121f392","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca4d30784529b8868121f392: start","job_status":"start","queue_duration_s":0.521599,"scheduling_latency_s":0.521613}2158gitlab # [ 259.637772] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca4d30784529b8868121f392","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca4d30784529b8868121f392: done: 0.015163 sec","job_status":"done","queue_duration_s":0.521599,"scheduling_latency_s":0.521613,"redis_calls":13,"redis_duration_s":0.002218,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001348,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015163,"completed_at":"2026-09-06T16:34:47.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2159gitlab # [ 259.691812] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e9cb84b562754b6bb9ff846","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e9cb84b562754b6bb9ff846: start","job_status":"start","queue_duration_s":0.539135,"scheduling_latency_s":0.539175}2160gitlab # [ 259.692639] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e9cb84b562754b6bb9ff846","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e9cb84b562754b6bb9ff846: done: 0.022431 sec","job_status":"done","queue_duration_s":0.539135,"scheduling_latency_s":0.539175,"redis_calls":13,"redis_duration_s":0.0029360000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022431,"completed_at":"2026-09-06T16:34:47.313Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2161gitlab # [ 259.735091] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08a823a7aed552586be5b9a2","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08a823a7aed552586be5b9a2: start","job_status":"start","queue_duration_s":0.563335,"scheduling_latency_s":0.563368}2162gitlab # [ 259.735842] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08a823a7aed552586be5b9a2","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08a823a7aed552586be5b9a2: done: 0.01665 sec","job_status":"done","queue_duration_s":0.563335,"scheduling_latency_s":0.563368,"redis_calls":13,"redis_duration_s":0.002234,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001477,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01665,"completed_at":"2026-09-06T16:34:47.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2163gitlab # [ 259.800984] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb9889eb6a3739848663521c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb9889eb6a3739848663521c: start","job_status":"start","queue_duration_s":0.580809,"scheduling_latency_s":0.580823}2164gitlab # [ 259.802121] gitaly[917]: time="2026-09-06T16:34:47.826Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:47.825" grpc.time_ms=0.681 pid=917 span.kind=server system=grpc2165gitlab # [ 259.802821] gitaly[917]: time="2026-09-06T16:34:47.857Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3889 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:47.856" grpc.time_ms=1.016 pid=917 span.kind=server system=grpc2166gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2167gitlab # [ 259.803493] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb9889eb6a3739848663521c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb9889eb6a3739848663521c: done: 0.008967 sec","job_status":"done","queue_duration_s":0.580809,"scheduling_latency_s":0.580823,"redis_calls":13,"redis_duration_s":0.001545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008967,"completed_at":"2026-09-06T16:34:47.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 259.863845] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6e997ecf8eff340191a753e","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6e997ecf8eff340191a753e: start","job_status":"start","queue_duration_s":0.590569,"scheduling_latency_s":0.590583}2169gitlab # [ 259.864983] gitaly[917]: time="2026-09-06T16:34:47.885Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:47.884" grpc.time_ms=0.773 pid=917 span.kind=server system=grpc2170gitlab # [ 259.865564] gitaly[917]: time="2026-09-06T16:34:47.895Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS diskcache=de492e06-e2d2-4372-8f8c-7a4c32639bd1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:17.801" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:47.802" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2171gitlab # [ 259.866174] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6e997ecf8eff340191a753e","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6e997ecf8eff340191a753e: done: 0.008804 sec","job_status":"done","queue_duration_s":0.590569,"scheduling_latency_s":0.590583,"redis_calls":13,"redis_duration_s":0.0015559999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008804,"completed_at":"2026-09-06T16:34:47.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 259.918080] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebde40bff755f8b4591ad18d","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebde40bff755f8b4591ad18d: start","job_status":"start","queue_duration_s":0.600213,"scheduling_latency_s":0.600226}2173gitlab # [ 259.918779] gitaly[917]: time="2026-09-06T16:34:47.895Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=85 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7462 command.oublock=0 command.real_time_ms=83 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:17.801" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:47.802" grpc.time_ms=93.127 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2174gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.10 seconds)2175gitlab # [ 259.919557] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebde40bff755f8b4591ad18d","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebde40bff755f8b4591ad18d: done: 0.008778 sec","job_status":"done","queue_duration_s":0.600213,"scheduling_latency_s":0.600226,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008778,"completed_at":"2026-09-06T16:34:47.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2176gitlab # [ 259.969228] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b391835c0f2b6c0e5f8d87a2","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b391835c0f2b6c0e5f8d87a2: start","job_status":"start","queue_duration_s":0.609808,"scheduling_latency_s":0.609822}2177gitlab # [ 259.969890] gitaly[917]: time="2026-09-06T16:34:47.940Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:17.839" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:47.939" grpc.time_ms=0.772 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2178gitlab # [ 259.970712] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 602179gitlab # [ 259.971221] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b391835c0f2b6c0e5f8d87a2","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b391835c0f2b6c0e5f8d87a2: done: 0.00921 sec","job_status":"done","queue_duration_s":0.609808,"scheduling_latency_s":0.609822,"redis_calls":13,"redis_duration_s":0.0015079999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00921,"completed_at":"2026-09-06T16:34:47.371Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab # [ 260.007817] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b402eda273cd8307c189ffa","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b402eda273cd8307c189ffa: start","job_status":"start","queue_duration_s":0.619836,"scheduling_latency_s":0.619849}2181gitlab # [ 260.052064] gitaly[917]: time="2026-09-06T16:34:47.975Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS51PG3A881R676QWFHNCS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:17.874" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:47.974" grpc.time_ms=0.567 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2182gitlab # [ 260.052815] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b402eda273cd8307c189ffa","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b402eda273cd8307c189ffa: done: 0.008741 sec","job_status":"done","queue_duration_s":0.619836,"scheduling_latency_s":0.619849,"redis_calls":13,"redis_duration_s":0.001478,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008741,"completed_at":"2026-09-06T16:34:47.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 260.065596] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96848c773859dc0da31c1846","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96848c773859dc0da31c1846: start","job_status":"start","queue_duration_s":0.629396,"scheduling_latency_s":0.629409}2184gitlab # [ 260.076786] gitaly[917]: time="2026-09-06T16:34:48.056Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS532DFQKGR8Q4RFR1GS80 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:18.054" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:48.055" grpc.time_ms=0.496 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2185gitlab # [ 260.117051] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96848c773859dc0da31c1846","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96848c773859dc0da31c1846: done: 0.008814 sec","job_status":"done","queue_duration_s":0.629396,"scheduling_latency_s":0.629409,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008814,"completed_at":"2026-09-06T16:34:47.390Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186gitlab # [ 260.122722] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"304168486b66a117eb080a85","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-304168486b66a117eb080a85: start","job_status":"start","queue_duration_s":0.639023,"scheduling_latency_s":0.639037}2187gitlab # [ 260.164154] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"304168486b66a117eb080a85","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-304168486b66a117eb080a85: done: 0.008754 sec","job_status":"done","queue_duration_s":0.639023,"scheduling_latency_s":0.639037,"redis_calls":13,"redis_duration_s":0.0014939999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008754,"completed_at":"2026-09-06T16:34:47.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188gitlab # [ 260.168063] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a3aa4c4bd9e492730c1a47b","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a3aa4c4bd9e492730c1a47b: start","job_status":"start","queue_duration_s":0.648784,"scheduling_latency_s":0.648798}2189gitlab # [ 260.212559] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a3aa4c4bd9e492730c1a47b","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a3aa4c4bd9e492730c1a47b: done: 0.008949 sec","job_status":"done","queue_duration_s":0.648784,"scheduling_latency_s":0.648798,"redis_calls":13,"redis_duration_s":0.0016740000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008949,"completed_at":"2026-09-06T16:34:47.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 260.213868] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b114e7346e7e34653b589b34","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b114e7346e7e34653b589b34: start","job_status":"start","queue_duration_s":0.658802,"scheduling_latency_s":0.658815}2191gitlab # [ 260.225032] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b114e7346e7e34653b589b34","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b114e7346e7e34653b589b34: done: 0.008866 sec","job_status":"done","queue_duration_s":0.658802,"scheduling_latency_s":0.658815,"redis_calls":13,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008866,"completed_at":"2026-09-06T16:34:47.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2192gitlab # [ 260.294623] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bb2930f67b4919dbce12f35","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bb2930f67b4919dbce12f35: start","job_status":"start","queue_duration_s":0.668496,"scheduling_latency_s":0.668509}2193gitlab # [ 260.295551] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bb2930f67b4919dbce12f35","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bb2930f67b4919dbce12f35: done: 0.008827 sec","job_status":"done","queue_duration_s":0.668496,"scheduling_latency_s":0.668509,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008827,"completed_at":"2026-09-06T16:34:47.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2194gitlab # [ 260.340752] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60413473c5012d1da87f7583","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60413473c5012d1da87f7583: start","job_status":"start","queue_duration_s":0.678128,"scheduling_latency_s":0.678142}2195gitlab # [ 260.341683] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60413473c5012d1da87f7583","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60413473c5012d1da87f7583: done: 0.012357 sec","job_status":"done","queue_duration_s":0.678128,"scheduling_latency_s":0.678142,"redis_calls":13,"redis_duration_s":0.0017299999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012357,"completed_at":"2026-09-06T16:34:47.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196gitlab # [ 260.391996] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcb3e7d93c4f6ddb7d42d9d5","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcb3e7d93c4f6ddb7d42d9d5: start","job_status":"start","queue_duration_s":0.691542,"scheduling_latency_s":0.691557}2197gitlab # [ 260.392725] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcb3e7d93c4f6ddb7d42d9d5","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcb3e7d93c4f6ddb7d42d9d5: done: 0.009269 sec","job_status":"done","queue_duration_s":0.691542,"scheduling_latency_s":0.691557,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009269,"completed_at":"2026-09-06T16:34:47.453Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2198gitlab # [ 260.433355] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa27b775a67a6cfb7bdd7e9c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa27b775a67a6cfb7bdd7e9c: start","job_status":"start","queue_duration_s":0.701641,"scheduling_latency_s":0.701654}2199gitlab # [ 260.434070] gitaly[917]: time="2026-09-06T16:34:48.484Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS523DH0RPKKKBMK03B85E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:58.077" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:48.477" grpc.time_ms=6.46 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 260.434990] nixbot[2660]: {"ts": "2026-09-06T16:34:48.514Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 \"HTTP/1.1 201 Created\""}2201gitlab # [ 260.435798] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 14922202gitlab # [ 260.436441] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa27b775a67a6cfb7bdd7e9c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa27b775a67a6cfb7bdd7e9c: done: 0.008852 sec","job_status":"done","queue_duration_s":0.701641,"scheduling_latency_s":0.701654,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008852,"completed_at":"2026-09-06T16:34:47.462Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203gitlab # [ 260.476349] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99269d750e82e86bb57ce15","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99269d750e82e86bb57ce15: start","job_status":"start","queue_duration_s":0.711503,"scheduling_latency_s":0.711517}2204gitlab # [ 260.477059] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99269d750e82e86bb57ce15","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99269d750e82e86bb57ce15: done: 0.008866 sec","job_status":"done","queue_duration_s":0.711503,"scheduling_latency_s":0.711517,"redis_calls":13,"redis_duration_s":0.001606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008866,"completed_at":"2026-09-06T16:34:47.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2205gitlab # [ 260.542695] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcab3567d3138d1042eb2ac4","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcab3567d3138d1042eb2ac4: start","job_status":"start","queue_duration_s":0.721194,"scheduling_latency_s":0.721207}2206gitlab # [ 260.543592] gitaly[917]: time="2026-09-06T16:34:48.567Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:18.066" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:48.566" grpc.time_ms=0.559 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2207gitlab # [ 260.544194] nixbot[2660]: {"ts": "2026-09-06T16:34:48.643Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 \"HTTP/1.1 201 Created\""}2208gitlab # [ 260.544672] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcab3567d3138d1042eb2ac4","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcab3567d3138d1042eb2ac4: done: 0.008732 sec","job_status":"done","queue_duration_s":0.721194,"scheduling_latency_s":0.721207,"redis_calls":13,"redis_duration_s":0.0014459999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008732,"completed_at":"2026-09-06T16:34:47.482Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209gitlab # [ 260.602520] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2443a43cda20a209e2a2ac2c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2443a43cda20a209e2a2ac2c: start","job_status":"start","queue_duration_s":0.730748,"scheduling_latency_s":0.73076}2210gitlab # [ 260.603195] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0984b3f14362b0077b7998aa4bc82cc88650f4d3 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1142211gitlab # [ 260.603812] gitaly[917]: time="2026-09-06T16:34:48.571Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS523DH0RPKKKBMK03B85E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:18.569" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:48.570" grpc.time_ms=0.654 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2212gitlab # [ 260.605262] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2443a43cda20a209e2a2ac2c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2443a43cda20a209e2a2ac2c: done: 0.009165 sec","job_status":"done","queue_duration_s":0.730748,"scheduling_latency_s":0.73076,"redis_calls":13,"redis_duration_s":0.0016589999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009165,"completed_at":"2026-09-06T16:34:47.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213gitlab # [ 260.646744] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dd71eb41cb342cce3e7da77","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dd71eb41cb342cce3e7da77: start","job_status":"start","queue_duration_s":0.740907,"scheduling_latency_s":0.740921}2214gitlab # [ 260.647595] gitaly[917]: time="2026-09-06T16:34:48.595Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS523DH0RPKKKBMK03B85E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:18.594" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:48.595" grpc.time_ms=0.515 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2215gitlab # [ 260.654791] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dd71eb41cb342cce3e7da77","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dd71eb41cb342cce3e7da77: done: 0.01239 sec","job_status":"done","queue_duration_s":0.740907,"scheduling_latency_s":0.740921,"redis_calls":13,"redis_duration_s":0.002497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01239,"completed_at":"2026-09-06T16:34:47.505Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 260.696305] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cfa926cb873330daaf43f7e","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cfa926cb873330daaf43f7e: start","job_status":"start","queue_duration_s":0.75454,"scheduling_latency_s":0.754562}2217gitlab # [ 260.702283] gitaly[917]: time="2026-09-06T16:34:48.600Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:34:58.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:48.594" grpc.time_ms=6.363 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2218gitlab # [ 260.714949] gitaly[917]: time="2026-09-06T16:34:48.721Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:48.720" grpc.time_ms=0.403 pid=917 span.kind=server system=grpc2219gitlab # [ 260.755883] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cfa926cb873330daaf43f7e","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cfa926cb873330daaf43f7e: done: 0.011275 sec","job_status":"done","queue_duration_s":0.75454,"scheduling_latency_s":0.754562,"redis_calls":13,"redis_duration_s":0.002243,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001349,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011275,"completed_at":"2026-09-06T16:34:47.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 260.758558] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e635f9d9bcae2c7257f41554","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e635f9d9bcae2c7257f41554: start","job_status":"start","queue_duration_s":0.7667,"scheduling_latency_s":0.766716}2221gitlab # [ 260.769773] gitaly[917]: time="2026-09-06T16:34:48.738Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3889 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:48.737" grpc.time_ms=0.444 pid=917 span.kind=server system=grpc2222gitlab # [ 260.812913] gitaly[917]: time="2026-09-06T16:34:48.758Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3891 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:34:48.757" grpc.time_ms=0.315 pid=917 span.kind=server system=grpc2223gitlab # [ 260.813630] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e635f9d9bcae2c7257f41554","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e635f9d9bcae2c7257f41554: done: 0.009619 sec","job_status":"done","queue_duration_s":0.7667,"scheduling_latency_s":0.766716,"redis_calls":13,"redis_duration_s":0.001756,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001246,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009619,"completed_at":"2026-09-06T16:34:47.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 260.825149] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6148c4d1f671cd8cd016d73a","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6148c4d1f671cd8cd016d73a: start","job_status":"start","queue_duration_s":0.777156,"scheduling_latency_s":0.777169}2225gitlab # [ 260.866391] gitaly[917]: time="2026-09-06T16:34:48.770Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S diskcache=19f543db-735f-4b72-9599-6c1f1abcdbe2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:18.698" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:48.698" pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2226gitlab # [ 260.867141] gitaly[917]: time="2026-09-06T16:34:48.770Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7081 command.oublock=0 command.real_time_ms=64 command.spawn_token_fork_ms=0 command.system_time_ms=38 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:35:18.698" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:34:48.698" grpc.time_ms=71.905 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2227gitlab # [ 260.867939] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6148c4d1f671cd8cd016d73a","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6148c4d1f671cd8cd016d73a: done: 0.009106 sec","job_status":"done","queue_duration_s":0.777156,"scheduling_latency_s":0.777169,"redis_calls":13,"redis_duration_s":0.001551,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009106,"completed_at":"2026-09-06T16:34:47.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2228gitlab # [ 260.908416] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11d8270e8da85c27dd74ca58","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11d8270e8da85c27dd74ca58: start","job_status":"start","queue_duration_s":0.787096,"scheduling_latency_s":0.787111}2229gitlab # [ 260.909080] gitaly[917]: time="2026-09-06T16:34:48.819Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:35:18.817" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:48.817" grpc.time_ms=0.733 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2230gitlab # [ 260.914219] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:48 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1442231gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses2232gitlab # [ 260.916163] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11d8270e8da85c27dd74ca58","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11d8270e8da85c27dd74ca58: done: 0.009223 sec","job_status":"done","queue_duration_s":0.787096,"scheduling_latency_s":0.787111,"redis_calls":13,"redis_duration_s":0.00169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009223,"completed_at":"2026-09-06T16:34:47.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 260.957109] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae9407a0674d54bcd684a63c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae9407a0674d54bcd684a63c: start","job_status":"start","queue_duration_s":0.797136,"scheduling_latency_s":0.79715}2234gitlab # [ 260.965622] gitaly[917]: time="2026-09-06T16:34:48.842Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS53JFVGB4T5R2ZG13GK5S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:35:18.841" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:34:48.842" grpc.time_ms=0.541 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2235gitlab # [ 261.004443] gitlab-workhorse[826]: time="2026-09-06T16:34:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672236gitlab # [ 261.005114] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 482237gitlab # [ 261.005618] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/09/06:16:34:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 482238gitlab # [ 261.006123] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae9407a0674d54bcd684a63c","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae9407a0674d54bcd684a63c: done: 0.009326 sec","job_status":"done","queue_duration_s":0.797136,"scheduling_latency_s":0.79715,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009326,"completed_at":"2026-09-06T16:34:47.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2239gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/0984b3f14362b0077b7998aa4bc82cc88650f4d3/statuses, in 0.09 seconds)2240(finished: subtest: eval and build post commit statuses, in 6.98 seconds)2241gitlab # [ 261.013185] sidekiq-cluster[2544]: {"severity":"INFO","time":"2026-09-06T16:34:47.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c290a59baa6396513e40e534","created_at":"2026-09-06T16:34:46.708Z","trace_propagation_headers":{"sentry-trace":"a4a4b7c4d43845afaeb4bc48c63d064a-e522f817b8564fd8","baggage":"sentry-trace_id=a4a4b7c4d43845afaeb4bc48c63d064a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c41b439dacffc14d7a795c0372b12464","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:34:46.752Z","job_size_bytes":2,"pid":2544,"sidekiq_tid":"m7xg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c290a59baa6396513e40e534: start","job_status":"start","queue_duration_s":0.807343,"scheduling_latency_s":0.807356}2242(finished: run the VM test script, in 309.73 seconds)2243gitlab # [ 261.059925] gitaly[917]: time="2026-09-06T16:34:48.979Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS53W33S0E51SDPD5C87JC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:34:48.972" grpc.time_ms=7.149 pid=917 remote_ip="2001:db8:1::1" span.kind=server system=grpc2244test script finished in 309.75s2245cleanup2246kill QemuMachine (pid 46)2247gitlab # [ 261.060695] gitaly[917]: time="2026-09-06T16:34:49.028Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS540NS27CKSXW0WTSX1HJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.requestqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2248(finished: cleanup, in 1.58 seconds)2249additionally exposed symbols:2250 gitlab,2251 vlan1,2252 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2253[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-06T16:33:51.566Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2254[]2255[]2256[]2257[]2258[]2259[]2260[]2261[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-06T16:34:40.476892Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2262[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2263[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2264[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2265[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2266[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': '2026-09-06T16:34:46.942Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2267[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': '2026-09-06T16:34:46.942Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2268[{'id': 5, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-09-06T16:34:40.706Z', 'started_at': None, 'finished_at': '2026-09-06T16:34:46.942Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '0984b3f14362b0077b7998aa4bc82cc88650f4d3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-06T16:34:48.498Z', 'started_at': None, 'finished_at': '2026-09-06T16:34:48.617Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]