vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #443
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.I2VHtqiyow', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: fd9acf92-a268-491c-9580-536695a3dfe416gitlab # Superblock backups stored on blocks:17gitlab # 32768, 98304, 163840, 22937618gitlab # 19gitlab # Allocating group tables: 0/8 done20gitlab # Writing inode tables: 0/8 done21gitlab # Creating journal (8192 blocks): done22gitlab # Writing superblocks and filesystem accounting information: 0/8 done23gitlab # 24gitlab # Virtualisation disk image created.25gitlab # Creating Nix store image...26(finished: start all VMs, in 75.65 seconds)27gitlab # Created Nix store image.28gitlab: waiting for unit setup-gitlab.service29gitlab: waiting for the VM to finish booting30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/knq5zdh7zcv4gjd8z6qlrg8l8mg8fc5n-nixos-system-gitlab-test/init regInfo=/nix/store/rq6wmlfwq1f59cpfhr5s9q53xf3vqd0r-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 455539393 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000812] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000880] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002740] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002751] Using GB pages for direct mapping73gitlab # [ 0.002855] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002862] ACPI: Early table checksum verification disabled75gitlab # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002869] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002873] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002879] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002882] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002883] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002885] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002886] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002888] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002889] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002889] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002890] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002890] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002891] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003106] No NUMA configuration found90gitlab # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003109] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.005442] Zone ranges:93gitlab # [ 0.005442] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005444] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005445] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005446] Device empty97gitlab # [ 0.005447] Movable zone start for each node98gitlab # [ 0.005447] Early memory node ranges99gitlab # [ 0.005447] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005448] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005449] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005451] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005478] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005748] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.116810] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.117230] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.117245] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.117275] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.117278] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.117280] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.117281] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.117282] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.117282] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.117285] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.117286] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.117290] TSC deadline timer available117gitlab # [ 0.117295] CPU topo: Max. logical packages: 1118gitlab # [ 0.117296] CPU topo: Max. logical dies: 1119gitlab # [ 0.117296] CPU topo: Max. dies per package: 1120gitlab # [ 0.117299] CPU topo: Max. threads per core: 1121gitlab # [ 0.117300] CPU topo: Num. cores per package: 4122gitlab # [ 0.117300] CPU topo: Num. threads per package: 4123gitlab # [ 0.117301] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.117324] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.117341] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.117343] kvm-guest: setup PV sched yield127gitlab # [ 0.117358] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.117360] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.117361] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.117362] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.117364] Booting paravirtualized kernel on KVM132gitlab # [ 0.117367] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.121830] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.124075] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.124123] kvm-guest: PV spinlocks enabled136gitlab # [ 0.124124] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.124127] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/knq5zdh7zcv4gjd8z6qlrg8l8mg8fc5n-nixos-system-gitlab-test/init regInfo=/nix/store/rq6wmlfwq1f59cpfhr5s9q53xf3vqd0r-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.124222] Unknown kernel command line parameters "regInfo=/nix/store/rq6wmlfwq1f59cpfhr5s9q53xf3vqd0r-closure-info/registration", will be passed to user space.139gitlab # [ 0.124235] random: crng init done140gitlab # [ 0.124236] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.133249] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.137911] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.137984] software IO TLB: area num 4.144gitlab # [ 0.213805] Fallback order for Node 0: 0145gitlab # [ 0.213812] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.213813] Policy zone: Normal147gitlab # [ 0.216334] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.225908] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.240112] allocated 12582912 bytes of page_ext150gitlab # [ 0.250148] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.250150] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.251001] Dynamic Preempt: lazy153gitlab # [ 0.251188] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.251189] rcu: RCU event tracing is enabled.155gitlab # [ 0.251189] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.251191] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.251191] Rude variant of Tasks RCU enabled.158gitlab # [ 0.251192] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.251192] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.251193] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.251220] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.251222] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.251223] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.255618] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.255909] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.255917] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.256028] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.259615] Console: colour VGA+ 80x25169gitlab # [ 0.259618] printk: legacy console [tty0] enabled170gitlab # [ 0.290541] printk: legacy console [ttyS0] enabled171gitlab # [ 0.398588] ACPI: Core revision 20250807172gitlab # [ 0.399542] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.401145] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.402160] x2apic enabled175gitlab # [ 0.402935] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.403905] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.405207] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.406706] kvm-guest: setup PV IPIs179gitlab # [ 0.408293] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.409350] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.411106] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.413191] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.414231] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.416103] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.417107] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.418104] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.419104] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.420103] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.421862] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.422907] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.424111] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.425104] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.426104] active return thunk: srso_alias_return_thunk194gitlab # [ 0.427125] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.428103] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.429922] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.430929] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.432103] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.433103] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.434104] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.435104] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.436103] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.437103] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.438907] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.439908] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.440908] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.470435] Freeing SMP alternatives memory: 44K208gitlab # [ 0.471106] pid_max: default: 32768 minimum: 301209gitlab # [ 0.472197] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.473197] landlock: Up and running.211gitlab # [ 0.473862] Yama: becoming mindful.212gitlab # [ 0.474770] LSM support for eBPF active213gitlab # [ 0.475312] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.477252] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.479833] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.480695] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.481108] ... version: 2218gitlab # [ 0.481880] ... bit width: 48219gitlab # [ 0.482106] ... generic counters: 6220gitlab # [ 0.482863] ... generic bitmap: 000000000000003f221gitlab # [ 0.483162] ... fixed-purpose counters: 0222gitlab # [ 0.483938] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.484857] ... value mask: 0000ffffffffffff224gitlab # [ 0.485106] ... max period: 00007fffffffffff225gitlab # [ 0.486033] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.486964] signal: max sigframe size: 3376227gitlab # [ 0.487713] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.488109] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.489187] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.494623] smp: Bringing up secondary CPUs ...231gitlab # [ 0.495505] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.496114] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.499243] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.500844] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.501447] Memory: 6011568K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.503106] devtmpfs: initialized237gitlab # [ 0.504199] x86/mm: Memory block size: 128MB238gitlab # [ 0.508152] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.509184] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.510197] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.511457] PM: RTC time: 10:44:37, date: 2026-10-10242gitlab # [ 0.515221] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.517447] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.518161] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.520224] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.521118] audit: initializing netlink subsys (disabled)247gitlab # [ 0.522161] audit: type=2000 audit(1791629078.201:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.522406] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.524107] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.525106] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.526106] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.527106] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.528120] cpuidle: using governor menu254gitlab # [ 0.530343] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.531338] PCI: Using configuration type 1 for base access256gitlab # [ 0.532107] PCI: Using configuration type 1 for extended access257gitlab # [ 0.533298] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.536163] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.537107] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.538106] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.539106] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.544291] ACPI: Added _OSI(Module Device)263gitlab # [ 0.546114] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.548114] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.555111] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.561366] ACPI: Interpreter enabled267gitlab # [ 0.562125] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.562893] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.563307] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.567115] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.569799] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.583785] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.586121] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.591277] acpiphp: Slot [3] registered275gitlab # [ 0.592178] acpiphp: Slot [4] registered276gitlab # [ 0.594162] acpiphp: Slot [5] registered277gitlab # [ 0.596184] acpiphp: Slot [6] registered278gitlab # [ 0.597149] acpiphp: Slot [7] registered279gitlab # [ 0.599150] acpiphp: Slot [8] registered280gitlab # [ 0.600148] acpiphp: Slot [9] registered281gitlab # [ 0.602168] acpiphp: Slot [10] registered282gitlab # [ 0.603150] acpiphp: Slot [11] registered283gitlab # [ 0.604149] acpiphp: Slot [12] registered284gitlab # [ 0.606164] acpiphp: Slot [13] registered285gitlab # [ 0.607159] acpiphp: Slot [14] registered286gitlab # [ 0.609149] acpiphp: Slot [15] registered287gitlab # [ 0.610147] acpiphp: Slot [16] registered288gitlab # [ 0.611191] acpiphp: Slot [17] registered289gitlab # [ 0.613150] acpiphp: Slot [18] registered290gitlab # [ 0.614147] acpiphp: Slot [19] registered291gitlab # [ 0.616148] acpiphp: Slot [20] registered292gitlab # [ 0.616976] acpiphp: Slot [21] registered293gitlab # [ 0.617635] acpiphp: Slot [22] registered294gitlab # [ 0.618126] acpiphp: Slot [23] registered295gitlab # [ 0.618810] acpiphp: Slot [24] registered296gitlab # [ 0.619136] acpiphp: Slot [25] registered297gitlab # [ 0.619837] acpiphp: Slot [26] registered298gitlab # [ 0.621127] acpiphp: Slot [27] registered299gitlab # [ 0.621831] acpiphp: Slot [28] registered300gitlab # [ 0.622126] acpiphp: Slot [29] registered301gitlab # [ 0.622845] acpiphp: Slot [30] registered302gitlab # [ 0.624126] acpiphp: Slot [31] registered303gitlab # [ 0.624851] PCI host bridge to bus 0000:00304gitlab # [ 0.625117] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.626107] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.627106] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.628106] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.630119] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.633185] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.635780] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.640441] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.645182] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.652182] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.654180] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.657112] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.659111] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.662112] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.664594] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.671312] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.676615] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.679650] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.682122] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.684581] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.702157] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.703120] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.704128] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.705846] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.709372] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.720155] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.723129] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.725168] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.756130] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.761362] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.769140] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.770065] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.770924] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.773429] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.778129] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.779066] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.779916] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.783656] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.788144] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.789805] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.791128] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.793357] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.797137] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.798064] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.800128] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.802464] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.806135] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.807036] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.807931] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.810910] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.814135] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.815052] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.815916] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.817093] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.820144] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.823147] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.824128] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.826345] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.830143] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.831063] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.832922] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.835355] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.838114] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.839168] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.839914] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.846860] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.847272] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.849245] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.850240] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.851181] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.854240] iommu: Default domain type: Translated375gitlab # [ 0.855108] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.856335] ACPI: bus type USB registered377gitlab # [ 0.857173] usbcore: registered new interface driver usbfs378gitlab # [ 0.858130] usbcore: registered new interface driver hub379gitlab # [ 0.859037] usbcore: registered new device driver usb380gitlab # [ 0.861185] NetLabel: Initializing381gitlab # [ 0.861818] NetLabel: domain hash size = 128382gitlab # [ 0.863107] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.864165] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.865115] PCI: Using ACPI for IRQ routing385gitlab # [ 0.866288] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.867102] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.867102] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.869116] vgaarb: loaded389gitlab # [ 0.869851] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.871106] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.876196] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.877815] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.878622] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.880169] pnp: PnP ACPI init395gitlab # [ 0.881310] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.892136] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.895091] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.897153] NET: Registered PF_INET protocol family399gitlab # [ 0.901208] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.920070] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.921647] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.923071] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.925705] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.927086] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.928761] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.930398] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.931842] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.933260] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.934318] NET: Registered PF_XDP protocol family410gitlab # [ 0.935217] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.936272] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.937336] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.938508] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.939671] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.941053] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.942074] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.944541] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.947102] PCI: CLS 0 bytes, default 64419gitlab # [ 0.947927] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.948710] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.949005] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.951868] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 0.972645] Initialise system trusted keyrings424gitlab # [ 0.973815] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.985143] Key type asymmetric registered426gitlab # [ 0.985894] Asymmetric key parser 'x509' registered427gitlab # [ 0.986784] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.988479] io scheduler mq-deadline registered429gitlab # [ 0.989300] io scheduler kyber registered430gitlab # [ 0.991839] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.993233] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.995602] Linux agpgart interface v0.103433gitlab # [ 0.996429] ACPI: bus type drm_connector registered434gitlab # [ 0.998897] usbcore: registered new interface driver usbserial_generic435gitlab # [ 1.000010] usbserial: USB Serial support registered for generic436gitlab # [ 1.001040] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 1.001040] Please enable it if your BIOS has the CPPC option.438gitlab # [ 1.003411] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 1.004856] drop_monitor: Initializing network drop monitor service440gitlab # [ 1.006036] NET: Registered PF_INET6 protocol family441gitlab # [ 1.007899] Segment Routing with IPv6442gitlab # [ 1.008608] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.010774] IPI shorthand broadcast: enabled444gitlab # [ 1.015750] sched_clock: Marking stable (868011458, 147185544)->(1037323873, -22126871)445gitlab # [ 1.017659] registered taskstats version 1446gitlab # [ 1.018714] Loading compiled-in X.509 certificates447gitlab # [ 1.029061] Demotion targets for Node 0: null448gitlab # [ 1.030166] Key type .fscrypt registered449gitlab # [ 1.030884] Key type fscrypt-provisioning registered450gitlab # [ 1.031844] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.032789] ima: Allocated hash algorithm: sha1452gitlab # [ 1.033645] ima: No architecture policies found453gitlab # [ 1.034703] PM: Magic number: 14:519:729454gitlab # [ 1.036367] RAS: Correctable Errors collector initialized.455gitlab # [ 1.040842] clk: Disabling unused clocks456gitlab # [ 1.041585] PM: genpd: Disabling unused power domains457gitlab # [ 1.253442] Freeing initrd memory: 29392K458gitlab # [ 1.257130] Freeing unused decrypted memory: 2028K459gitlab # [ 1.260082] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.261242] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.263419] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.265081] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.316617] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.317782] Run /init as init process465gitlab # [ 1.331321] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.356091] fuse: init (API version 7.45)467gitlab # [ 1.364420] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.371135] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.374250] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.416034] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.752910] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.793536] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.795692] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.797853] systemd[1]: Running in initrd.475gitlab # [ 1.800425] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.803352] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.916652] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.963145] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.973816] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.974994] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.976485] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.977681] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.978830] systemd[1]: Reached target Path Units.484gitlab # [ 1.979720] systemd[1]: Reached target Slice Units.485gitlab # [ 1.980636] systemd[1]: Reached target Swaps.486gitlab # [ 1.981487] systemd[1]: Reached target Timer Units.487gitlab # [ 1.982501] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.983742] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.984917] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.985921] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.986937] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.987886] systemd[1]: Reached target Socket Units.493gitlab # [ 1.989816] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.991982] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 1.994113] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 1.999203] systemd[1]: Starting Journal Service...497gitlab # [ 2.003308] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 2.004345] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 2.009285] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 2.011534] systemd[1]: Finished Create List of Static Device Nodes.501gitlab # [ 2.020278] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...502gitlab # [ 2.029993] systemd-journald[92]: Collecting audit messages is disabled.503gitlab # [ 2.036301] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.504gitlab # [ 2.038723] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev505gitlab # [ 2.040671] systemd[1]: modprobe@configfs.service: Deactivated successfully.506gitlab # [ 2.042358] systemd[1]: Finished Load Kernel Module configfs.507gitlab # [ 2.050046] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config508gitlab # [ 2.059144] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.509gitlab # [ 2.062258] systemd[1]: Starting Create Static Device Nodes in /dev...510gitlab # [ 2.066625] netfs: FS-Cache loaded511gitlab # [ 2.073911] 9pnet: Installing 9P2000 support512gitlab # [ 2.075635] systemd[1]: Finished Load Kernel Modules.513gitlab # [ 2.079385] systemd[1]: Starting Apply Kernel Variables...514gitlab # [ 1.935518] systemd-modules-load[93]: Using 4 probe threads515gitlab # [ 2.083436] systemd[1]: Started Journal Service.516gitlab # [ 1.936983] systemd-modules-load[93]: Inserted module 'virtio_balloon'517gitlab # [ 1.938498] systemd-modules-load[93]: Inserted module 'dm_mod'518gitlab # [ 1.939587] systemd-modules-load[93]: Inserted module 'virtio_gpu'519gitlab # [ 1.940679] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.520gitlab # [ 1.941941] systemd[1]: Finished Load Kernel Module 9pnet_virtio.521gitlab # [ 1.943410] systemd[1]: Finished Create Static Device Nodes in /dev.522gitlab # [ 1.944628] systemd[1]: Reached target Preparation for Local File Systems.523gitlab # [ 1.945646] systemd[1]: Reached target Local File Systems.524gitlab # [ 1.946489] systemd[1]: Starting Create System Files and Directories...525gitlab # [ 1.947528] systemd[1]: Starting Rule-based Manager for Device Events and Files...526gitlab # [ 1.960189] systemd[1]: Finished Apply Kernel Variables.527gitlab # [ 1.961485] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 1.972669] systemd-udevd[111]: Using default interface naming scheme 'v261'.529gitlab # [ 1.983340] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 2.014573] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.015529] systemd[1]: Reached target System Initialization.532gitlab # [ 2.016438] systemd[1]: Reached target Basic System.533gitlab # [ 2.244066] uhci_hcd 0000:00:01.2: UHCI Host Controller534gitlab # [ 2.244961] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1535gitlab # [ 2.246130] uhci_hcd 0000:00:01.2: detected 2 ports536gitlab # [ 2.247020] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140537gitlab # [ 2.248161] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18538gitlab # [ 2.249478] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1539gitlab # [ 2.250642] usb usb1: Product: UHCI Host Controller540gitlab # [ 2.251487] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd541gitlab # [ 2.252638] usb usb1: SerialNumber: 0000:00:01.2542gitlab # [ 2.260838] virtio_blk virtio4: 4/0/0 default/read/poll queues543gitlab # [ 2.260882] hub 1-0:1.0: USB hub found544gitlab # [ 2.262671] hub 1-0:1.0: 2 ports detected545gitlab # [ 2.265987] SCSI subsystem initialized546gitlab # [ 2.272736] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12547gitlab # [ 2.273638] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)548gitlab # [ 2.277081] serio: i8042 KBD port at 0x60,0x64 irq 1549gitlab # [ 2.277842] serio: i8042 AUX port at 0x60,0x64 irq 12550gitlab # [ 2.279721] virtio_blk virtio5: 4/0/0 default/read/poll queues551gitlab # [ 2.282412] virtio_blk virtio5: [vdb] 11953160 512-byte logical blocks (6.12 GB/5.70 GiB)552gitlab # [ 2.179479] systemd[1]: Starting Virtual Console Setup...553gitlab # [ 2.338500] scsi host0: ata_piix554gitlab # [ 2.339395] scsi host1: ata_piix555gitlab # [ 2.340022] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0556gitlab # [ 2.341267] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0557gitlab # [ 2.194742] (udev-worker)[139]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.558gitlab # [ 2.199440] (udev-worker)[139]: Network interface NamePolicy= disabled on kernel command line.559gitlab # [ 2.203312] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line.560gitlab # [ 2.205341] systemd-vconsole-setup[145]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.207290] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 2.216133] systemd[1]: Found device /dev/disk/by-label/nixos.563gitlab # [ 2.218343] systemd[1]: Found device /dev/disk/by-label/nix-store.564gitlab # [ 2.219248] systemd[1]: Reached target Initrd Root Device.565gitlab # [ 2.220072] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...566gitlab # [ 2.244900] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks567gitlab # [ 2.248454] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.568gitlab # [ 2.491399] usb 1-1: new full-speed USB device number 2 using uhci_hcd569gitlab # [ 2.507517] ata2: found unknown device (class 0)570gitlab # [ 2.510293] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100571gitlab # [ 2.514509] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5572gitlab # [ 2.544670] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0573gitlab # [ 2.579770] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray574gitlab # [ 2.608068] cdrom: Uniform CD-ROM driver Revision: 3.20575gitlab # [ 2.659928] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.663226] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.665871] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.667566] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.669076] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.702413] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.713976] usbcore: registered new interface driver usbhid582gitlab # [ 2.716425] usbhid: USB HID core driver583gitlab # [ 2.724578] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.727861] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.867030] systemd[1]: Mounting /sysroot...586gitlab # [ 3.150489] EXT4-fs (vda): mounted filesystem fd9acf92-a268-491c-9580-536695a3dfe4 r/w with ordered data mode. Quota mode: none.587gitlab # [ 3.010744] systemd[1]: Mounted /sysroot.588gitlab # [ 3.012346] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 3.016293] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 3.018538] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.024825] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.028349] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.030292] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.039109] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.044822] systemd[1]: Mounted /sysroot/run.596gitlab # [ 3.196708] 9p: Installing v9fs 9p2000 file system support597gitlab # [ 3.051960] systemd[1]: Mounted /sysroot/nix/.rw-store.598gitlab # [ 3.201736] erofs (device vdb): mounted with root inode @ nid 128.599gitlab # [ 3.062652] systemd[1]: Starting rw-sysroot-nix-store.service...600gitlab # [ 3.064244] systemd[1]: Mounted /sysroot/nix/.ro-store.601gitlab # [ 3.065402] systemd[1]: Mounted /sysroot/tmp/shared.602gitlab # [ 3.066682] systemd-sysroot-fstab-check[178]: /sysroot should be mounted in the initrd, will request daemon-reload.603gitlab # [ 3.069089] systemd[1]: Reload requested from client PID 178 ('systemd-sysroot') (unit initrd-parse-etc.service)...604gitlab # [ 3.072165] systemd[1]: Reloading...605gitlab # [ 3.128598] systemd[1]: Reloading finished in 59 ms.606gitlab # [ 3.148517] systemd-sysroot-fstab-check[178]: Requesting initrd-fs.target/start/replace...607gitlab # [ 3.151502] systemd[1]: Mounted /sysroot/tmp/xchg.608gitlab # [ 3.153297] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.155739] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.157996] systemd-sysroot-fstab-check[178]: Requesting swap.target/start/replace...611gitlab # [ 3.160598] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.162657] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.164915] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.167212] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.171064] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.173509] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.867944] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.917548] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.920452] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.924319] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.927840] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.951469] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.956430] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.970588] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.972525] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.974571] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 4.011424] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 4.013557] systemd[1]: Stopped target Basic System.629gitlab # [ 4.015357] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 4.017261] systemd[1]: Stopped target Path Units.631gitlab # [ 4.018975] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 4.021609] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 4.026276] systemd[1]: Stopped target Slice Units.634gitlab # [ 4.030954] systemd[1]: Stopped target Socket Units.635gitlab # [ 4.033051] systemd[1]: Stopped target System Initialization.636gitlab # [ 4.035394] systemd[1]: Stopped target Swaps.637gitlab # [ 4.037227] systemd[1]: Stopped target Timer Units.638gitlab # [ 4.039050] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 4.041185] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 4.043235] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 4.046204] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.048938] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.052588] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.054301] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.056211] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.057735] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.059671] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.061169] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.063356] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.065504] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.067478] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.069193] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.070584] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.072180] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.073497] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.074360] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.075455] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.076528] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.077346] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.078294] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.079275] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.080365] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.081284] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.082257] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.083329] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.084527] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.085286] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.086132] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.087264] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.088203] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.089340] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.090426] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.091486] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.100689] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.101824] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.102978] systemd[1]: Reached target Switch Root.677gitlab # [ 4.104666] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.243065] initrd-nixos-activation-start[226]: booting system configuration /nix/store/knq5zdh7zcv4gjd8z6qlrg8l8mg8fc5n-nixos-system-gitlab-test679gitlab # [ 4.287063] initrd-nixos-activation-start[226]: running activation script...680gitlab # [ 4.547278] initrd-nixos-activation-start[249]: setting up /etc...681gitlab # [ 4.679368] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.682632] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.687821] systemd[1]: Starting Switch Root...684gitlab # [ 4.720567] systemd[1]: Switching root.685gitlab # [ 4.924949] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 5.036910] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.405421] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.414915] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.416889] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.418917] systemd[1]: Detected first boot.691gitlab # [ 5.421545] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.562839] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.646335] systemd[1]: Applying preset policy.694gitlab # [ 5.837304] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.953492] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.954890] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.957237] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.959552] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.961020] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.962633] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.963636] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.964918] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.966095] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.966862] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.967671] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.968593] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.969394] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.970278] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.971208] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.972247] systemd[1]: Reached target Path Units.711gitlab # [ 5.972998] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.973875] systemd[1]: Reached target Slice Units.713gitlab # [ 5.974673] systemd[1]: Reached target Swaps.714gitlab # [ 5.976141] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.978257] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.979681] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.981380] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.982391] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.984891] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.986383] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.988276] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.989526] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.990718] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.992133] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.995083] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.996215] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.997107] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.999954] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 6.002211] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 6.004388] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 6.007412] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 6.014443] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 6.015521] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 6.018734] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 6.019809] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 6.021367] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 6.023252] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 6.026075] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 6.027037] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 6.044603] systemd[1]: Starting Journal Service...741gitlab # [ 6.047898] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 6.061288] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 6.068240] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 6.069272] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.081356] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.083158] systemd-journald[321]: Collecting audit messages is enabled.747gitlab # [ 6.086942] loop: module loaded748gitlab # [ 6.089707] systemd[1]: Listening on Journal Log Access Socket.749gitlab # [ 6.090654] EXT4-fs (vda): re-mounted fd9acf92-a268-491c-9580-536695a3dfe4.750gitlab # [ 6.093300] systemd[1]: Mounted Huge Pages File System.751gitlab # [ 6.094404] systemd[1]: Mounted POSIX Message Queue File System.752gitlab # [ 6.096132] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.753gitlab # [ 6.097498] systemd[1]: Mounted Kernel Debug File System.754gitlab # [ 6.098496] systemd[1]: Mounted Kernel Trace File System.755gitlab # [ 5.954396] systemd[1]: Queued start job for default target Multi-User System.756gitlab # [ 6.101792] systemd[1]: Finished Create List of Static Device Nodes.757gitlab # [ 5.955967] systemd[1]: Unnecessary job was removed for /dev/vdb.758gitlab # [ 6.103963] systemd[1]: Started Journal Service.759gitlab # [ 5.957825] systemd[1]: systemd-journald.service: Deactivated successfully.760gitlab # [ 5.959753] systemd-modules-load[322]: Using 3 probe threads761gitlab # [ 5.960883] systemd-modules-load[322]: Inserted module 'loop'762gitlab # [ 5.962213] systemd-oomd[324]: No swap; memory pressure usage will be degraded763gitlab # [ 5.963763] systemd-modules-load[322]: Inserted module 'tls'764gitlab # [ 5.964831] systemd[1]: modprobe@configfs.service: Deactivated successfully.765gitlab # [ 5.965995] systemd[1]: Finished Load Kernel Module configfs.766gitlab # [ 5.966959] systemd[1]: Mounted FUSE Control File System.767gitlab # [ 5.967956] systemd[1]: Finished Load Kernel Modules.768gitlab # [ 5.968818] systemd[1]: Finished Remount Root and Kernel File Systems.769gitlab # [ 5.969786] systemd[1]: Listening on Disk Image Download Service Socket.770gitlab # [ 5.970755] systemd[1]: Mounting Kernel Configuration File System...771gitlab # [ 5.971728] systemd[1]: Starting Firewall...772gitlab # [ 5.972854] systemd[1]: Starting Flush Journal to Persistent Storage...773gitlab # [ 5.973875] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore774gitlab # [ 5.982936] systemd[1]: Starting Load/Save OS Random Seed...775gitlab # [ 5.984220] systemd[1]: Starting Apply Kernel Variables...776gitlab # [ 5.988866] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...777gitlab # [ 6.137514] systemd-journald[321]: Received client request to flush runtime journal.778gitlab # [ 6.035370] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os779gitlab # [ 6.037108] systemd[1]: Mounted Kernel Configuration File System.780gitlab # [ 6.038262] systemd[1]: Finished Load/Save OS Random Seed.781gitlab # [ 6.039318] systemd[1]: Reached target First Boot Complete.782gitlab # [ 6.040802] systemd[1]: Finished Apply Kernel Variables.783gitlab # [ 6.041708] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.784gitlab # [ 6.042797] systemd[1]: Starting Create Static Device Nodes in /dev...785gitlab # [ 6.043792] systemd[1]: Finished Flush Journal to Persistent Storage.786gitlab # [ 6.045191] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 6.046082] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 6.047095] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 6.072727] systemd-udevd[365]: Using default interface naming scheme 'v261'.790gitlab # [ 6.093156] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.103408] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.187410] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.201575] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.240086] (udev-worker)[434]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.795gitlab # [ 6.242307] (udev-worker)[434]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 6.259141] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.797gitlab # [ 6.263693] systemd[1]: Condition check resulted in Virtio network device being skipped.798gitlab # [ 6.265954] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore799gitlab # [ 6.267766] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67800gitlab # [ 6.271078] (udev-worker)[421]: Network interface NamePolicy= disabled on kernel command line.801gitlab # [ 6.272496] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore802gitlab # [ 6.274151] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os803gitlab # [ 6.275453] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.433660] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3805gitlab # [ 6.450276] ACPI: button: Power Button [PWRF]806gitlab # [ 6.462294] rtc_cmos 00:05: RTC can wake from S4807gitlab # [ 6.322072] systemd[1]: Finished Firewall.808gitlab # [ 6.490080] rtc_cmos 00:05: registered as rtc0809gitlab # [ 6.491280] rtc_cmos 00:05: setting system clock to 2026-10-10T10:44:43 UTC (1791629083)810gitlab # [ 6.492882] mousedev: PS/2 mouse device common for all mice811gitlab # [ 6.493014] parport_pc 00:03: reported by Plug and Play ACPI812gitlab # [ 6.494832] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]813gitlab # [ 6.496079] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs814gitlab # [ 6.498410] Floppy drive(s): fd0 is 2.88M AMI BIOS815gitlab # [ 6.500340] systemd-journald[321]: Time jumped backwards, rotating.816gitlab # [ 6.512676] FDC 0 is a S82078B817gitlab # [ 6.530193] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5818gitlab # [ 6.531205] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0819gitlab # [ 6.531963] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4820gitlab # [ 6.537256] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD821gitlab # [ 6.553985] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console822gitlab # [ 6.571044] Console: switching to colour dummy device 80x25823gitlab # [ 6.571955] [drm] Found bochs VGA, ID 0xb0c5.824gitlab # [ 6.571958] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.825gitlab # [ 6.573694] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic826gitlab # [ 6.574944] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0827gitlab # [ 6.594068] Console: switching to colour frame buffer device 160x50828gitlab # [ 6.596901] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device829gitlab # [ 6.463344] systemd[1]: Starting Virtual Console Setup...830gitlab # [ 6.619008] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6831gitlab # [ 6.662537] kvm_amd: TSC scaling supported832gitlab # [ 6.663004] kvm_amd: Nested Virtualization enabled833gitlab # [ 6.663606] kvm_amd: Nested Paging enabled834gitlab # [ 6.664036] kvm_amd: LBR virtualization supported835gitlab # [ 6.664595] kvm_amd: Virtual VMLOAD VMSAVE supported836gitlab # [ 6.665132] kvm_amd: Virtual GIF supported837gitlab # [ 6.665662] kvm_amd: Virtual NMI enabled838gitlab # [ 6.527847] systemd-vconsole-setup[491]: Configuration of first virtual console was skipped, ignoring remaining ones.839gitlab # [ 6.530098] systemd[1]: Finished Virtual Console Setup.840gitlab # [ 6.539417] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.841gitlab # [ 6.540727] systemd[1]: Stopped Virtual Console Setup.842gitlab # [ 6.541478] systemd[1]: Stopping Virtual Console Setup...843gitlab # [ 6.548437] systemd[1]: Starting Virtual Console Setup...844gitlab # [ 6.563437] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.845gitlab # [ 6.566764] systemd[1]: Stopped Virtual Console Setup.846gitlab # [ 6.577958] systemd[1]: Starting Virtual Console Setup...847gitlab # [ 6.729505] ppdev: user-space parallel port driver848gitlab # [ 6.730329] EDAC MC: Ver: 3.0.0849gitlab # [ 6.599639] systemd-vconsole-setup[498]: Configuration of first virtual console was skipped, ignoring remaining ones.850gitlab # [ 6.601255] systemd[1]: Finished Virtual Console Setup.851gitlab # [ 6.811202] systemd[1]: Mounting /run/wrappers...852gitlab # [ 6.837826] systemd[1]: Mounted /run/wrappers.853gitlab # [ 6.839356] systemd[1]: Reached target Local File Systems.854gitlab # [ 6.841177] systemd[1]: Listening on Boot Loader Control Service Socket.855gitlab # [ 6.843918] systemd[1]: Starting register-nix-paths.service...856gitlab # [ 6.846617] systemd[1]: Starting Create SUID/SGID Wrappers...857gitlab # [ 6.847925] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.858gitlab # [ 6.849647] systemd[1]: Starting Save Transient machine-id to Disk...859gitlab # [ 6.868629] systemd[1]: Starting Create System Files and Directories...860gitlab # [ 6.893990] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.861gitlab # [ 6.896622] systemd[1]: Finished Save Transient machine-id to Disk.862gitlab # [ 6.910312] systemd[1]: Finished Create System Files and Directories.863gitlab # [ 6.913541] systemd[1]: Starting Rebuild Journal Catalog...864gitlab # [ 6.915501] systemd[1]: Starting Record System Boot/Shutdown in UTMP...865gitlab # [ 6.938164] systemd[1]: Finished Record System Boot/Shutdown in UTMP.866gitlab # [ 6.941900] systemd[1]: Finished Rebuild Journal Catalog.867gitlab # [ 6.943680] systemd[1]: Starting Update is Completed...868gitlab # [ 6.962306] systemd[1]: Finished Update is Completed.869gitlab # [ 7.006594] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.870gitlab # [ 7.007673] systemd[1]: Finished Create SUID/SGID Wrappers.871gitlab # [ 7.234863] systemd[1]: Finished register-nix-paths.service.872gitlab # [ 7.236492] systemd[1]: Reached target System Initialization.873gitlab # [ 7.237855] systemd[1]: Started Discard unused filesystem blocks once a week.874gitlab # [ 7.239391] systemd[1]: Started logrotate.timer.875gitlab # [ 7.240613] systemd[1]: Started Daily Cleanup of Temporary Directories.876gitlab # [ 7.242183] systemd[1]: Reached target Timer Units.877gitlab # [ 7.243427] systemd[1]: Listening on D-Bus System Message Bus Socket.878gitlab # [ 7.244856] systemd[1]: Listening on Nix Daemon Socket.879gitlab # [ 7.248354] systemd[1]: Starting nixbot CI service socket...880gitlab # [ 7.249679] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.881gitlab # [ 7.251515] systemd[1]: Listening on nixbot CI service socket.882gitlab # [ 7.252867] systemd[1]: Reached target Socket Units.883gitlab # [ 7.254801] systemd[1]: Starting D-Bus System Message Bus...884gitlab # [ 7.327493] dbus-broker-launch[580]: Looking up NSS user entry for 'systemd-timesync'...885gitlab # [ 7.329563] dbus-broker-launch[580]: NSS returned no entry for 'systemd-timesync'886gitlab # [ 7.331332] dbus-broker-launch[580]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"887gitlab # [ 7.334314] systemd[1]: Started D-Bus System Message Bus.888gitlab # [ 7.348367] dbus-broker-launch[580]: Ready889gitlab # [ 7.351836] systemd[1]: Reached target Basic System.890gitlab # [ 7.354930] systemd[1]: Started backdoor.service.891gitlab # [ 7.357635] systemd[1]: Starting gitlab-config.service...892gitlab # [ 7.362422] systemd[1]: Starting Import lastlog data into lastlog2 database...893gitlab # [ 7.376155] systemd[1]: Starting Logrotate configuration check...894gitlab # [ 7.383233] systemd[1]: Starting Name Service Cache Daemon (nsncd)...895gitlab # [ 7.390645] systemd[1]: Starting Post-Boot Actions...896gitlab # [ 7.395173] systemd[1]: Started Reset console on configuration changes.897gitlab # connecting to host...898gitlab # [ 7.399851] logrotate[585]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.899gitlab # [ 7.401924] logrotate[585]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf900gitlab # [ 7.406231] logrotate[585]: note: 'monthly' overrides previously specified 'weekly'901gitlab # [ 7.407758] logrotate[585]: Reading state from file: /var/lib/logrotate.status902gitlab # [ 7.408918] logrotate[585]: state file /var/lib/logrotate.status does not exist903gitlab: Guest shell says: b'Spawning backdoor root shell...\n'904gitlab # [ 7.410749] logrotate[585]: Allocating hash table for state file, size 64 entries905gitlab # [ 7.411846] logrotate[585]: Handling 4 logs906gitlab: connected to guest root shell907gitlab # [ 7.412413] logrotate[585]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed908gitlab: (connecting took 8.02 seconds)909gitlab # [ 7.414390] logrotate[585]: considering log /var/log/btmp910gitlab: (finished: waiting for the VM to finish booting, in 8.02 seconds)911gitlab # [ 7.415065] logrotate[585]: Creating new state912gitlab # [ 7.415644] logrotate[585]: Now: 2026-10-10 10:44913gitlab # [ 7.416279] logrotate[585]: Last rotated at 2026-10-10 10:00914gitlab # [ 7.417022] logrotate[585]: log does not need rotating (log has already been rotated)915gitlab # [ 7.418038] logrotate[585]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed916gitlab # [ 7.421501] logrotate[585]: considering log /var/log/wtmp917gitlab # [ 7.422184] logrotate[585]: Creating new state918gitlab # [ 7.422658] logrotate[585]: Now: 2026-10-10 10:44919gitlab # [ 7.423303] logrotate[585]: Last rotated at 2026-10-10 10:00920gitlab # [ 7.423873] logrotate[585]: log does not need rotating (log has already been rotated)921gitlab # [ 7.429159] logrotate[585]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed922gitlab # [ 7.430880] logrotate[585]: switching euid from 0 to 165 and egid from 0 to 165 (pid 585)923gitlab # [ 7.431933] logrotate[585]: considering log /var/gitlab/state/log/*.log924gitlab # [ 7.432764] logrotate[585]: log /var/gitlab/state/log/*.log does not exist -- skipping925gitlab # [ 7.433844] logrotate[585]: Creating new state926gitlab # [ 7.434465] logrotate[585]: switching euid from 165 to 0 and egid from 165 to 0 (pid 585)927gitlab # [ 7.435638] logrotate[585]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed928gitlab # [ 7.437294] logrotate[585]: switching euid from 0 to 60 and egid from 0 to 60 (pid 585)929gitlab # [ 7.438404] logrotate[585]: considering log /var/log/nginx/*.log930gitlab # [ 7.439253] logrotate[585]: log /var/log/nginx/*.log does not exist -- skipping931gitlab # [ 7.440245] logrotate[585]: Creating new state932gitlab # [ 7.440902] logrotate[585]: not running postrotate script, since no logs were rotated933gitlab # [ 7.442020] logrotate[585]: switching euid from 60 to 0 and egid from 60 to 0 (pid 585)934gitlab # [ 7.443094] systemd[1]: Starting resolvconf update...935gitlab # [ 7.444345] nsncd[586]: Oct 10 10:44:44.587 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"936gitlab # [ 7.445302] systemd[1]: Finished Logrotate configuration check.937gitlab # [ 7.446088] systemd[1]: Finished Post-Boot Actions.938gitlab # [ 7.446898] systemd[1]: Started Name Service Cache Daemon (nsncd).939gitlab # [ 7.447833] systemd[1]: Reached target Host and Network Name Lookups.940gitlab # [ 7.448468] systemd[1]: Reached target User and Group Name Lookups.941gitlab # [ 7.449101] systemd[1]: Starting User Login Management...942gitlab # [ 7.449758] systemd[1]: Finished Import lastlog data into lastlog2 database.943gitlab # [ 7.485624] systemd-logind[622]: New seat seat0.944gitlab # [ 7.489883] systemd[1]: Stopped target Host and Network Name Lookups.945gitlab # [ 7.492777] systemd[1]: Stopping Host and Network Name Lookups...946gitlab # [ 7.493481] systemd[1]: Stopped target User and Group Name Lookups.947gitlab # [ 7.493955] systemd[1]: Stopping User and Group Name Lookups...948gitlab # [ 7.494722] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...949gitlab # [ 7.495448] systemd-logind[622]: Watching system buttons on /dev/input/event2 (Power Button)950gitlab # [ 7.496087] systemd-logind[622]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)951gitlab # [ 7.496640] systemd-logind[622]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)952gitlab # [ 7.497126] systemd[1]: nscd.service: Deactivated successfully.953gitlab # [ 7.497673] systemd[1]: Stopped Name Service Cache Daemon (nsncd).954gitlab # [ 7.498224] systemd[1]: nscd.service: Consumed 32ms CPU time over 108ms wall clock time, 7.4M memory peak, 5.5M read from disk.955gitlab # [ 7.498913] systemd[1]: Started User Login Management.956gitlab # [ 7.499847] systemd[1]: Starting linger-users.service...957gitlab # [ 7.501712] systemd[1]: Starting Name Service Cache Daemon (nsncd)...958gitlab # [ 7.516844] nsncd[678]: Oct 10 10:44:44.672 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"959gitlab # [ 7.519074] systemd[1]: Started Name Service Cache Daemon (nsncd).960gitlab # [ 7.520160] systemd[1]: linger-users.service: Deactivated successfully.961gitlab # [ 7.521102] systemd[1]: Finished linger-users.service.962gitlab # [ 7.521855] systemd[1]: Reached target Host and Network Name Lookups.963gitlab # [ 7.522895] systemd[1]: Reached target User and Group Name Lookups.964gitlab # [ 7.528569] systemd[1]: Finished resolvconf update.965gitlab # [ 7.529379] systemd[1]: Reached target Preparation for Network.966gitlab # [ 7.531141] systemd[1]: Starting DHCP Client...967gitlab # [ 7.533294] systemd[1]: Starting Address configuration of eth1...968gitlab # [ 7.535334] systemd[1]: Starting Extra networking commands....969gitlab # [ 7.560588] network-addresses-eth1-start[703]: adding address 192.168.1.1/24... done970gitlab # [ 7.564465] network-addresses-eth1-start[703]: adding address 2001:db8:1::1/64... done971gitlab # [ 7.575732] systemd[1]: Finished Address configuration of eth1.972gitlab # [ 7.607873] dhcpcd[712]: dhcpcd-10.3.2 starting973gitlab # [ 7.608892] systemd[1]: Finished Extra networking commands..974gitlab # [ 7.609878] systemd[1]: Reached target Network.975gitlab # [ 7.611754] systemd[1]: Starting gitlab-workhorse.service...976gitlab # [ 7.613912] dhcpcd[768]: dev: loaded udev977gitlab # [ 7.617067] systemd[1]: Starting Nginx Web Server...978gitlab # [ 7.623275] systemd[1]: Starting PostgreSQL Server...979gitlab # [ 7.771647] 8021q: 802.1Q VLAN Support v1.8980gitlab # [ 7.772592] 8021q: adding VLAN 0 to HW filter on device eth1981gitlab # [ 7.627804] systemd[1]: Starting Redis Server - redis-gitlab...982gitlab # [ 7.629364] systemd[1]: Starting Permit User Sessions...983gitlab # [ 7.642662] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.984gitlab # [ 7.654668] systemd[1]: Finished Permit User Sessions.985gitlab # [ 7.661608] systemd[1]: Started Getty on tty1.986gitlab # [ 7.663622] systemd[1]: Reached target Login Prompts.987gitlab # [ 7.875112] cfg80211: Loading compiled-in X.509 certificates for regulatory database988gitlab # [ 7.881259] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'989gitlab # [ 7.881998] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'990gitlab # [ 7.883410] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2991gitlab # [ 7.884464] cfg80211: failed to load regulatory.db992gitlab # [ 7.749649] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok993gitlab # [ 7.752159] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful994gitlab # [ 7.755181] systemd[1]: Started Nginx Web Server.995gitlab # [ 7.919155] 8021q: adding VLAN 0 to HW filter on device eth0996gitlab # [ 7.773727] dhcpcd[768]: eth0: waiting for carrier997gitlab # [ 7.774722] dhcpcd[768]: eth0: carrier acquired998gitlab # [ 7.780632] dhcpcd[768]: DUID 00:01:00:01:32:5c:d3:9c:52:54:00:12:34:56999gitlab # [ 7.781502] dhcpcd[768]: eth0: IAID 00:12:34:561000gitlab # [ 7.782427] dhcpcd[768]: eth0: adding address fe80::5054:ff:fe12:34561001gitlab # [ 7.805362] redis-server[792]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1002gitlab # [ 7.806057] redis-server[792]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1003gitlab # [ 7.806648] redis-server[792]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=792, just started1004gitlab # [ 7.807159] redis-server[792]: Configuration loaded1005gitlab # [ 7.807636] redis[792]: monotonic clock: POSIX clock_gettime1006gitlab # [ 7.808515] redis[792]: Running mode=standalone, port=0.1007gitlab # [ 7.809280] redis[792]: Server initialized1008gitlab # [ 7.809699] systemd[1]: Started Redis Server - redis-gitlab.1009gitlab # [ 7.810305] redis[792]: Ready to accept connections unix1010gitlab # [ 7.810736] redis[792]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1011gitlab # [ 7.827079] postgresql-pre-start[795]: The files belonging to this database system will be owned by user "postgres".1012gitlab # [ 7.827625] postgresql-pre-start[795]: This user must also own the server process.1013gitlab # [ 7.830652] postgresql-pre-start[795]: The database cluster will be initialized with locale "en_US.UTF-8".1014gitlab # [ 7.831587] postgresql-pre-start[795]: The default database encoding has accordingly been set to "UTF8".1015gitlab # [ 7.832101] postgresql-pre-start[795]: The default text search configuration will be set to "english".1016gitlab # [ 7.832580] postgresql-pre-start[795]: Data page checksums are enabled.1017gitlab # [ 7.833302] postgresql-pre-start[795]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1018gitlab # [ 7.833746] postgresql-pre-start[795]: creating subdirectories ... ok1019gitlab # [ 7.834415] postgresql-pre-start[795]: selecting dynamic shared memory implementation ... posix1020gitlab # [ 7.865650] postgresql-pre-start[795]: selecting default "max_connections" ... 1001021gitlab # [ 7.913371] postgresql-pre-start[795]: selecting default "shared_buffers" ... 128MB1022gitlab # [ 8.130237] systemd[1]: Started gitlab-workhorse.service.1023gitlab # [ 8.284134] postgresql-pre-start[795]: selecting default time zone ... UTC1024gitlab # [ 8.285897] postgresql-pre-start[795]: creating configuration files ... ok1025gitlab # [ 8.374659] gitlab-workhorse[828]: time="2026-10-10T10:44:45Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41026gitlab # [ 8.375675] gitlab-workhorse[828]: time="2026-10-10T10:44:45Z" level=info msg="Using redis/go-redis"1027gitlab # [ 8.376609] gitlab-workhorse[828]: time="2026-10-10T10:44:45Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1028gitlab # [ 8.463349] postgresql-pre-start[795]: running bootstrap script ... ok1029gitlab # [ 8.882429] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok1030gitlab # [ 9.133496] postgresql-pre-start[795]: syncing data to disk ... ok1031gitlab # [ 9.135128] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections1032gitlab # [ 9.136192] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1033gitlab # [ 9.137375] postgresql-pre-start[795]: Success. You can now start the database server using:1034gitlab # [ 9.138493] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1035gitlab # [ 9.260642] postgres[853]: [853] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1036gitlab # [ 9.264711] postgres[853]: [853] LOG: listening on IPv6 address "::1", port 54321037gitlab # [ 9.266356] postgres[853]: [853] LOG: listening on IPv4 address "127.0.0.1", port 54321038gitlab # [ 9.273244] postgres[853]: [853] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1039gitlab # [ 9.287965] postgres[862]: [862] LOG: database system was shut down at 2026-10-10 10:44:46 GMT1040gitlab # [ 9.302078] postgres[853]: [853] LOG: database system is ready to accept connections1041gitlab # [ 9.306207] systemd[1]: Started PostgreSQL Server.1042gitlab # [ 9.307825] systemd[1]: Starting PostgreSQL Setup Scripts...1043gitlab # [ 9.441748] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[866]: mkdir -p /var/gitlab/state/home/.ssh: OK1044gitlab # [ 9.445607] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[866]: chmod 700 /var/gitlab/state/home/.ssh: OK1045gitlab # [ 9.474105] dhcpcd[768]: eth0: soliciting an IPv6 router1046gitlab # [ 9.475317] dhcpcd[768]: eth0: Router Advertisement from fe80::21047gitlab # [ 9.476330] dhcpcd[768]: eth0: adding address fec0::5054:ff:fe12:3456/641048gitlab # [ 9.477411] dhcpcd[768]: eth0: adding route to fec0::/641049gitlab # [ 9.478208] dhcpcd[768]: eth0: adding default route via fe80::21050gitlab # [ 9.495205] systemd[1]: Finished gitlab-config.service.1051gitlab # [ 9.497531] systemd[1]: Started gitaly.service.1052gitlab # [ 9.544760] postgresql-setup-start[878]: CREATE DATABASE1053gitlab # [ 9.564194] postgresql-setup-start[906]: CREATE ROLE1054gitlab # [ 9.572043] postgresql-setup-start[908]: ALTER DATABASE1055gitlab # [ 9.590911] postgresql-setup-start[913]: CREATE ROLE1056gitlab # [ 9.592973] systemd[1]: Finished PostgreSQL Setup Scripts.1057gitlab # [ 9.594103] systemd[1]: Reached target PostgreSQL.1058gitlab # [ 9.595931] systemd[1]: Starting gitlab-db-config.service...1059gitlab # [ 9.597716] systemd[1]: Starting gitlab-postgresql.service...1060gitlab # [ 9.696202] dhcpcd[768]: eth0: soliciting a DHCP lease1061gitlab # [ 9.849379] NET: Registered PF_PACKET protocol family1062gitlab # [ 9.702426] gitlab-postgresql-start[923]: CREATE DATABASE1063gitlab # [ 9.708044] dhcpcd[768]: eth0: offered 10.0.2.15 from 10.0.2.21064gitlab # [ 9.711958] dhcpcd[768]: eth0: probing address 10.0.2.15/241065gitlab # [ 9.755435] gitlab-postgresql-start[934]: CREATE EXTENSION1066gitlab # [ 9.836410] gitlab-postgresql-start[940]: CREATE EXTENSION1067gitlab # [ 9.838731] systemd[1]: Finished gitlab-postgresql.service.1068gitlab # [ 10.027414] gitaly[892]: time="2026-10-10T10:44:47.182Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=8921069gitlab # [ 10.027809] gitaly[892]: time="2026-10-10T10:44:47.183Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=8921070gitlab # [ 10.028420] gitaly[892]: time="2026-10-10T10:44:47.183Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=8921071gitlab # [ 10.028924] gitaly[892]: time="2026-10-10T10:44:47.183Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=8921072gitlab # [ 10.029531] gitaly[892]: time="2026-10-10T10:44:47.183Z" level=info msg="Starting Gitaly" pid=892 version=1073gitlab # [ 10.230675] gitaly[892]: time="2026-10-10T10:44:47.386Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=201 pid=8921074gitlab # [ 10.232383] gitaly[892]: time="2026-10-10T10:44:47.387Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=8921075gitlab # [ 10.233237] gitaly[892]: time="2026-10-10T10:44:47.387Z" level=info msg="finished initializing command factory" duration_ms=0 pid=8921076gitlab # [ 10.233745] gitaly[892]: time="2026-10-10T10:44:47.387Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-892/git-exec-4286394343.d/git pid=8921077gitlab # [ 10.241799] gitaly[892]: time="2026-10-10T10:44:47.397Z" level=info msg="finished detecting git version" duration_ms=9 pid=8921078gitlab # [ 10.242377] gitaly[892]: time="2026-10-10T10:44:47.397Z" level=info msg="using Git version" pid=892 version=2.53.g1079gitlab # [ 10.243970] gitaly[892]: time="2026-10-10T10:44:47.398Z" level=info msg="clearing disk cache object folder" pid=892 storage=default1080gitlab # [ 10.245043] gitaly[892]: time="2026-10-10T10:44:47.399Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache249941493 pid=892 storage=default1081gitlab # [ 10.245560] gitaly[892]: time="2026-10-10T10:44:47.399Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache249941493 pid=892 storage=default1082gitlab # [ 10.246240] gitaly[892]: time="2026-10-10T10:44:47.399Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=8921083gitlab # [ 10.246774] gitaly[892]: time="2026-10-10T10:44:47.399Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=8921084gitlab # [ 10.247495] gitaly[892]: time="2026-10-10T10:44:47.399Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache249941493 clear_duration_ms=0 pid=892 storage=default1085gitlab # [ 10.277455] gitaly[892]: time="2026-10-10T10:44:47.433Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=8921086gitlab # [ 10.278050] gitaly[892]: time="2026-10-10T10:44:47.433Z" level=info msg="finished tempdir cleaner walk" pid=892 storage=default time_ms=01087gitlab # [ 10.278473] gitaly[892]: time="2026-10-10T10:44:47.433Z" level=info msg="counting repositories" pid=8921088gitlab # [ 10.278986] gitaly[892]: time="2026-10-10T10:44:47.433Z" level=info msg="starting to count repositories" pid=892 storage_path=/var/gitlab/state/repositories1089gitlab # [ 10.279683] gitaly[892]: time="2026-10-10T10:44:47.433Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=892 schema=unix1090gitlab # [ 10.280434] gitaly[892]: time="2026-10-10T10:44:47.433Z" level=info msg="listening at address" address=/run/gitaly/gitaly-892/sock.d/intern pid=892 schema=unix1091gitlab # [ 10.281162] gitaly[892]: time="2026-10-10T10:44:47.434Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=892 storage_path=/var/gitlab/state/repositories1092gitlab # [ 10.281777] gitaly[892]: time="2026-10-10T10:44:47.434Z" level=info msg="completed counting all repositories" counting_duration=0 pid=8921093gitlab # [ 10.292507] gitaly[892]: time="2026-10-10T10:44:47.448Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=8921094gitlab # [ 10.293344] gitaly[892]: time="2026-10-10T10:44:47.448Z" level=info msg="Started Gitaly" duration_ms=264 pid=8921095gitlab # [ 10.293883] gitaly[892]: time="2026-10-10T10:44:47.448Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=8921096gitlab # [ 10.294533] gitaly[892]: time="2026-10-10T10:44:47.448Z" level=info msg="maintenance: daily scheduled" pid=892 scheduled="2026-10-10 12:00:00 +0000 UTC"1097gitlab # [ 11.052553] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1098gitlab # [ 11.054266] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1099gitlab # [ 11.089665] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1100gitlab # [ 12.284822] gitaly[892]: time="2026-10-10T10:44:49.439Z" level=info msg="License database preloaded" duration_ms=2256 pid=8921101gitlab # [ 13.817242] dhcpcd[768]: eth0: leased 10.0.2.15 for 86400 seconds1102gitlab # [ 13.819303] dhcpcd[768]: eth0: adding route to 10.0.2.0/241103gitlab # [ 13.820985] dhcpcd[768]: eth0: adding default route via 10.0.2.21104gitlab # [ 13.939827] systemd[1]: Started DHCP Client.1105gitlab # [ 13.941959] systemd[1]: Reached target Network is Online.1106gitlab # [ 41.398378] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[918]: W, [2026-10-10T10:45:18.553341 #918] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1107gitlab # [ 43.929456] postgres[1321]: [1321] ERROR: relation "feature_gates" does not exist at character 5231108gitlab # [ 43.931803] postgres[1321]: [1321] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1109gitlab # [ 43.932491] postgres[1321]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1110gitlab # [ 43.934392] postgres[1321]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1111gitlab # [ 43.935949] postgres[1321]: attidentity AS identity,1112gitlab # [ 43.938334] postgres[1321]: attgenerated as attgenerated1113gitlab # [ 43.940260] postgres[1321]: FROM pg_attribute a1114gitlab # [ 43.941417] postgres[1321]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1115gitlab # [ 43.943452] postgres[1321]: LEFT JOIN pg_type t ON a.atttypid = t.oid1116gitlab # [ 43.944888] postgres[1321]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1117gitlab # [ 43.947263] postgres[1321]: WHERE a.attrelid = '"feature_gates"'::regclass1118gitlab # [ 43.949208] postgres[1321]: AND a.attnum > 0 AND NOT a.attisdropped1119gitlab # [ 43.949967] postgres[1321]: ORDER BY a.attnum1120gitlab # [ 43.951862] postgres[1321]:1121gitlab # [ 47.543403] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1363]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1122gitlab # [ 47.548116] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1363]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1123gitlab # [ 48.436366] gitlab-workhorse[828]: time="2026-10-10T10:45:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1124gitlab # [ 58.437523] gitlab-workhorse[828]: time="2026-10-10T10:45:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 63.378765] postgres[1438]: [1438] ERROR: relation "table_without_models" does not exist1126gitlab # [ 63.381706] postgres[1438]: [1438] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1127gitlab # [ 68.438339] gitlab-workhorse[828]: time="2026-10-10T10:45:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1128gitlab # [ 75.498199] systemd[1]: Finished gitlab-db-config.service.1129gitlab # [ 75.504798] systemd[1]: Started gitlab-sidekiq.service.1130gitlab # [ 75.508911] systemd[1]: Starting gitlab.service...1131gitlab # [ 75.849975] bundle[1605]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1132gitlab # [ 75.850787] bundle[1605]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1133gitlab # [ 75.851489] bundle[1605]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1134gitlab # [ 78.448780] gitlab-workhorse[828]: time="2026-10-10T10:45:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1135gitlab # [ 78.670841] sidekiq-cluster[1604]: {"severity":"INFO","time":"2026-10-10T10:45:55.808Z","message":"Starting cluster with 1 processes"}1136gitlab # [ 78.733634] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.866Z","pid":1605,"message":"* Enabling systemd notification integration"}1137gitlab # [ 78.734714] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"Puma starting in cluster mode..."}1138gitlab # [ 78.735771] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1139gitlab # [ 78.736678] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1140gitlab # [ 78.737450] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Min threads: 0"}1141gitlab # [ 78.738186] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Max threads: 4"}1142gitlab # [ 78.738904] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Environment: production"}1143gitlab # [ 78.739792] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Master PID: 1605"}1144gitlab # [ 78.740690] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Workers: 2"}1145gitlab # [ 78.741536] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1146gitlab # [ 78.742302] bundle[1605]: {"timestamp":"2026-10-10T10:45:55.888Z","pid":1605,"message":"* Preloading application"}1147gitlab # [ 88.458818] gitlab-workhorse[828]: time="2026-10-10T10:46:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1148gitlab # [ 98.469902] gitlab-workhorse[828]: time="2026-10-10T10:46:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1149gitlab # [ 102.234384] bundle[1605]: W, [2026-10-10T10:46:19.389414 #1605] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1150gitlab # [ 104.442110] sidekiq-cluster[1635]: 2026-10-10T10:46:21.597Z pid=1635 tid=wjv INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1635, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>447905792}1151gitlab # [ 104.616938] sidekiq-cluster[1635]: W, [2026-10-10T10:46:21.772643 #1635] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1152gitlab # [ 104.660698] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:21.816Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1153gitlab # [ 106.993634] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.148Z","message":"GitLab reliable fetch activated!"}1154gitlab # [ 106.997643] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.148Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1155gitlab # [ 107.014529] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.170Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1156gitlab # [ 107.019734] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.171Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1157gitlab # [ 107.024963] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.173Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1158gitlab # [ 107.025575] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.175Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1159gitlab # [ 107.026717] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.177Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1160gitlab # [ 107.027809] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.179Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1161gitlab # [ 107.044352] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.200Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1162gitlab # [ 107.050884] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.206Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1163gitlab # [ 107.058805] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.214Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1164gitlab # [ 107.061167] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.216Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1165gitlab # [ 107.063348] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.219Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1166gitlab # [ 107.065853] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.221Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1167gitlab # [ 107.067835] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.223Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1168gitlab # [ 107.070202] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.226Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1169gitlab # [ 107.072298] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.228Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1170gitlab # [ 107.074255] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.230Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1171gitlab # [ 107.076821] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.232Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1172gitlab # [ 107.079472] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.235Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1173gitlab # [ 107.082079] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.237Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1174gitlab # [ 107.084505] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.240Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1175gitlab # [ 107.087105] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.242Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1176gitlab # [ 107.089525] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.245Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1177gitlab # [ 107.091604] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.247Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1178gitlab # [ 107.093765] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.249Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1179gitlab # [ 107.096116] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.251Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1180gitlab # [ 107.098608] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.254Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1181gitlab # [ 107.100987] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.256Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1182gitlab # [ 107.103758] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.259Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1183gitlab # [ 107.108039] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.263Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1184gitlab # [ 107.110767] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.266Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1185gitlab # [ 107.113073] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.268Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1186gitlab # [ 107.115650] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.271Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1187gitlab # [ 107.118093] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.273Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1188gitlab # [ 107.120721] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.276Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1189gitlab # [ 107.122919] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.278Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1190gitlab # [ 107.124982] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.280Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1191gitlab # [ 107.127204] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.283Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1192gitlab # [ 107.129514] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.285Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1193gitlab # [ 107.131981] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.287Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1194gitlab # [ 107.134148] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.289Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1195gitlab # [ 107.136258] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.292Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1196gitlab # [ 107.138323] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.294Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1197gitlab # [ 107.141782] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.297Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1198gitlab # [ 107.145155] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.300Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1199gitlab # [ 107.148226] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.303Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1200gitlab # [ 107.171981] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.326Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1201gitlab # [ 107.175723] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.331Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1202gitlab # [ 107.179296] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.335Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1203gitlab # [ 107.182979] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.338Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1204gitlab # [ 107.187319] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.343Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1205gitlab # [ 107.190916] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.346Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1206gitlab # [ 107.195277] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.351Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1207gitlab # [ 107.206370] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.362Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1208gitlab # [ 107.210822] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.366Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1209gitlab # [ 107.214513] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.370Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1210gitlab # [ 107.218733] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.374Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1211gitlab # [ 107.222435] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.378Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1212gitlab # [ 107.228833] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.384Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1213gitlab # [ 107.233124] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.388Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1214gitlab # [ 107.238590] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.394Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1215gitlab # [ 107.243083] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.398Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1216gitlab # [ 107.247751] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.403Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1217gitlab # [ 107.252874] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.408Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1218gitlab # [ 107.258819] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.414Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1219gitlab # [ 107.264261] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.420Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1220gitlab # [ 107.269531] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.425Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1221gitlab # [ 107.274225] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.430Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1222gitlab # [ 107.278395] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.434Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1223gitlab # [ 107.283329] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.439Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1224gitlab # [ 107.287638] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.443Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1225gitlab # [ 107.291954] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.447Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1226gitlab # [ 107.296857] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.452Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1227gitlab # [ 107.301228] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.456Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1228gitlab # [ 107.305679] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.461Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1229gitlab # [ 107.310483] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.466Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1230gitlab # [ 107.315192] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.470Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1231gitlab # [ 107.320371] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.476Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1232gitlab # [ 107.325678] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.480Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1233gitlab # [ 107.329976] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.485Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1234gitlab # [ 107.334240] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.490Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1235gitlab # [ 107.338722] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.494Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1236gitlab # [ 107.343670] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.499Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1237gitlab # [ 107.349023] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.504Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1238gitlab # [ 107.353784] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.509Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1239gitlab # [ 107.358400] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.514Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1240gitlab # [ 107.363220] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.519Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1241gitlab # [ 107.368424] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.524Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1242gitlab # [ 107.373337] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.529Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1243gitlab # [ 107.377811] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.533Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1244gitlab # [ 107.382340] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.538Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1245gitlab # [ 107.386977] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.542Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1246gitlab # [ 107.391385] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.547Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1247gitlab # [ 107.395853] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.551Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1248gitlab # [ 107.400509] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.556Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1249gitlab # [ 107.405733] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.561Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1250gitlab # [ 107.411333] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.567Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1251gitlab # [ 107.414956] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.570Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1252gitlab # [ 107.418532] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.574Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1253gitlab # [ 107.426927] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.582Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1254gitlab # [ 107.430351] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.586Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1255gitlab # [ 107.433850] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.589Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1256gitlab # [ 107.438294] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.594Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1257gitlab # [ 107.442353] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.598Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1258gitlab # [ 107.444571] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.600Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1259gitlab # [ 107.448661] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.604Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1260gitlab # [ 107.452267] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.608Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1261gitlab # [ 107.455591] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.611Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1262gitlab # [ 107.459848] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.615Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1263gitlab # [ 107.463457] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:24.619Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1264gitlab # [ 108.470105] gitlab-workhorse[828]: time="2026-10-10T10:46:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1265gitlab # [ 118.471619] gitlab-workhorse[828]: time="2026-10-10T10:46:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1266gitlab # [ 128.473266] gitlab-workhorse[828]: time="2026-10-10T10:46:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1267gitlab # [ 130.233552] postgres[2100]: [2100] ERROR: relation "table_without_models" does not exist1268gitlab # [ 130.234890] postgres[2100]: [2100] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1269gitlab # [ 130.799452] postgres[2121]: [2121] ERROR: relation "table_without_models" does not exist1270gitlab # [ 130.800693] postgres[2121]: [2121] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1271gitlab # [ 132.650629] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:49.805Z","message":"Booted Rails 7.2.3.1 application in production environment"}1272gitlab # [ 132.653661] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:49.809Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1273gitlab # [ 132.655992] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:49.811Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1274gitlab # [ 132.657731] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:49.811Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1275gitlab # [ 132.660590] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:49.816Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1276gitlab # [ 133.477658] bundle[1605]: {"timestamp":"2026-10-10T10:46:50.632Z","pid":1605,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1277gitlab # [ 133.481919] bundle[1605]: {"timestamp":"2026-10-10T10:46:50.632Z","pid":1605,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1278gitlab # [ 133.485700] bundle[1605]: {"timestamp":"2026-10-10T10:46:50.632Z","pid":1605,"message":"! #\u003cThread:0x00007c6e9f7f3b38 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1279gitlab # [ 133.497158] bundle[1605]: {"timestamp":"2026-10-10T10:46:50.632Z","pid":1605,"message":"! #\u003cThread:0x00007c6e9f9e9dc0 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1280gitlab # [ 133.508368] bundle[1605]: {"timestamp":"2026-10-10T10:46:50.632Z","pid":1605,"message":"Use Ctrl-C to stop"}1281gitlab # [ 133.511301] bundle[1605]: {"timestamp":"2026-10-10T10:46:50.633Z","pid":1605,"message":"Sending status to systemd every 1.0 sec"}1282gitlab # [ 134.139355] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:51.294Z","message":"Cleaning working queues"}1283gitlab # [ 134.144853] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:51.300Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3302b6b5832edc2e15c644bc","created_at":"2026-10-10T10:45:52.211Z","trace_propagation_headers":{"sentry-trace":"da9ff820771e40cc9259456dbae3c9cd-07e4ed538d394e46","baggage":"sentry-trace_id=da9ff820771e40cc9259456dbae3c9cd,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d620e59aa51a162e4e6c6d8560d81457","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:45:52.219Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3302b6b5832edc2e15c644bc: start","job_status":"start","queue_duration_s":59.080821,"scheduling_latency_s":59.08089}1284gitlab # [ 134.212641] bundle[1605]: {"timestamp":"2026-10-10T10:46:51.367Z","pid":1605,"message":"- Worker 1 (PID: 2182) booted in 0.67s, phase: 0"}1285gitlab # [ 134.220402] bundle[1605]: {"timestamp":"2026-10-10T10:46:51.376Z","pid":1605,"message":"- Worker 0 (PID: 2178) booted in 0.7s, phase: 0"}1286gitlab # [ 134.224237] systemd[1]: Started gitlab.service.1287gitlab # [ 134.224927] systemd[1]: Reached target Common target for all GitLab services..1288gitlab # [ 134.227375] systemd[1]: Starting setup-gitlab.service...1289gitlab # [ 134.245280] setup-gitlab-start[2212]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1290gitlab # [ 134.295519] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:46:51.450Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"3302b6b5832edc2e15c644bc","created_at":"2026-10-10T10:45:52.211Z","trace_propagation_headers":{"sentry-trace":"da9ff820771e40cc9259456dbae3c9cd-07e4ed538d394e46","baggage":"sentry-trace_id=da9ff820771e40cc9259456dbae3c9cd,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d620e59aa51a162e4e6c6d8560d81457","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:45:52.219Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-3302b6b5832edc2e15c644bc: done: 0.150966 sec","job_status":"done","queue_duration_s":59.080821,"scheduling_latency_s":59.08089,"redis_calls":26,"redis_duration_s":0.003873,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.001965,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.069766689,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.150966,"completed_at":"2026-10-10T10:46:51.450Z","load_balancing_strategy":"primary","db_duration_s":0.034214,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1291gitlab # [ 136.153482] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:46:53 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 18911292gitlab # [ 136.158614] setup-gitlab-start[2212]: + TOKEN=nixbot-test-token-of-20-chars1293gitlab # [ 136.159346] setup-gitlab-start[2212]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1294gitlab # [ 136.173766] runuser[2231]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1295gitlab # [ 136.561834] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1296gitlab # [ 136.562842] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1297gitlab # [ 136.563789] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1298gitlab # [ 137.135126] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1299gitlab # [ 137.136311] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1300gitlab # [ 137.137289] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1301gitlab # [ 145.076454] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"03046bff8ae4d88a5636cd66","created_at":"2026-10-10T10:47:02.195Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26522b8e5f9eeb98f18e8a9c0c255fa0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.228Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-03046bff8ae4d88a5636cd66: start","job_status":"start","queue_duration_s":0.002985,"scheduling_latency_s":0.003033}1302gitlab # [ 145.350087] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"03046bff8ae4d88a5636cd66","created_at":"2026-10-10T10:47:02.195Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26522b8e5f9eeb98f18e8a9c0c255fa0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.228Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-03046bff8ae4d88a5636cd66: done: 0.273913 sec","job_status":"done","queue_duration_s":0.002985,"scheduling_latency_s":0.003033,"redis_calls":14,"redis_duration_s":0.017156,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004596,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003835,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000747023,"exclusive_lock_hold_duration_s":0.123376295,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.273913,"completed_at":"2026-10-10T10:47:02.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 145.416908] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"80b6a3664db4ee8833c398ea","created_at":"2026-10-10T10:47:02.274Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a1b05c8fd2e2104a43c90d68159ae03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.293Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-80b6a3664db4ee8833c398ea: start","job_status":"start","queue_duration_s":0.278594,"scheduling_latency_s":0.278646}1304gitlab # [ 145.491343] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.646Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1305gitlab # [ 145.509771] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"80b6a3664db4ee8833c398ea","created_at":"2026-10-10T10:47:02.274Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a1b05c8fd2e2104a43c90d68159ae03","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.293Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-80b6a3664db4ee8833c398ea: done: 0.09281 sec","job_status":"done","queue_duration_s":0.278594,"scheduling_latency_s":0.278646,"redis_calls":10,"redis_duration_s":0.03467,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00348,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.030036,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000618794,"duration_s":0.09281,"completed_at":"2026-10-10T10:47:02.664Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 145.569682] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.671Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8e3ba1ed9e32bc15461e4a0e","created_at":"2026-10-10T10:47:02.338Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5f5a26ff01cb6a9b3d4b1bf5108740","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.346Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8e3ba1ed9e32bc15461e4a0e: start","job_status":"start","queue_duration_s":0.324624,"scheduling_latency_s":0.324718}1307gitlab # [ 145.582920] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.727Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1308gitlab # [ 145.590917] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.745Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8e3ba1ed9e32bc15461e4a0e","created_at":"2026-10-10T10:47:02.338Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5f5a26ff01cb6a9b3d4b1bf5108740","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.346Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8e3ba1ed9e32bc15461e4a0e: done: 0.074908 sec","job_status":"done","queue_duration_s":0.324624,"scheduling_latency_s":0.324718,"redis_calls":10,"redis_duration_s":0.015023000000000002,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008951,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003614,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002825778,"duration_s":0.074908,"completed_at":"2026-10-10T10:47:02.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 145.612678] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"804d194787c146b7d7a27756","created_at":"2026-10-10T10:47:02.351Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06ae7f332116d113de88bc53c9609559","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.360Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-804d194787c146b7d7a27756: start","job_status":"start","queue_duration_s":0.405265,"scheduling_latency_s":0.405367}1310gitlab # [ 145.621730] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.777Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1311gitlab # [ 145.627365] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"804d194787c146b7d7a27756","created_at":"2026-10-10T10:47:02.351Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06ae7f332116d113de88bc53c9609559","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.360Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-804d194787c146b7d7a27756: done: 0.01708 sec","job_status":"done","queue_duration_s":0.405265,"scheduling_latency_s":0.405367,"redis_calls":10,"redis_duration_s":0.001317,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270146,"duration_s":0.01708,"completed_at":"2026-10-10T10:47:02.782Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 145.671589] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.783Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0e324b62b0f9c9c010b791ca","created_at":"2026-10-10T10:47:02.383Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"840229e40cd3341209a77179560e6fe5","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.399Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0e324b62b0f9c9c010b791ca: start","job_status":"start","queue_duration_s":0.384597,"scheduling_latency_s":0.384612}1313gitlab # [ 145.672550] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.808Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0e324b62b0f9c9c010b791ca","created_at":"2026-10-10T10:47:02.383Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"840229e40cd3341209a77179560e6fe5","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.399Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0e324b62b0f9c9c010b791ca: done: 0.024791 sec","job_status":"done","queue_duration_s":0.384597,"scheduling_latency_s":0.384612,"redis_calls":16,"redis_duration_s":0.00219,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001175,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212317,"duration_s":0.024791,"completed_at":"2026-10-10T10:47:02.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 145.694547] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"891f476819d5dda36fa2de68","created_at":"2026-10-10T10:47:02.408Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac379d7e7ee67e83574bc1e8dab8bfd9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.415Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-891f476819d5dda36fa2de68: start","job_status":"start","queue_duration_s":0.394303,"scheduling_latency_s":0.394317}1315gitlab # [ 145.742833] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.823Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"891f476819d5dda36fa2de68","created_at":"2026-10-10T10:47:02.408Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac379d7e7ee67e83574bc1e8dab8bfd9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.415Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-891f476819d5dda36fa2de68: done: 0.01408 sec","job_status":"done","queue_duration_s":0.394303,"scheduling_latency_s":0.394317,"redis_calls":12,"redis_duration_s":0.001605,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218743,"duration_s":0.01408,"completed_at":"2026-10-10T10:47:02.823Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 145.747325] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.824Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3245ac11c66ffcdab18d249e","created_at":"2026-10-10T10:47:02.431Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"430dfbc18c775d12d4a797fd7310abb1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.449Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3245ac11c66ffcdab18d249e: start","job_status":"start","queue_duration_s":0.375192,"scheduling_latency_s":0.375205}1317gitlab # [ 145.758565] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3245ac11c66ffcdab18d249e","created_at":"2026-10-10T10:47:02.431Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"430dfbc18c775d12d4a797fd7310abb1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.449Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3245ac11c66ffcdab18d249e: done: 0.024343 sec","job_status":"done","queue_duration_s":0.375192,"scheduling_latency_s":0.375205,"redis_calls":10,"redis_duration_s":0.001464,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000287746,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.024343,"completed_at":"2026-10-10T10:47:02.848Z","load_balancing_strategy":"primary","db_duration_s":0.007698,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 145.814058] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"95e33a48f143aa57f4994730","created_at":"2026-10-10T10:47:02.454Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.467Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-95e33a48f143aa57f4994730: start","job_status":"start","queue_duration_s":0.382472,"scheduling_latency_s":0.382487}1319gitlab # [ 145.814692] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"95e33a48f143aa57f4994730","created_at":"2026-10-10T10:47:02.454Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.467Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-95e33a48f143aa57f4994730: done: 0.038572 sec","job_status":"done","queue_duration_s":0.382472,"scheduling_latency_s":0.382487,"redis_calls":19,"redis_duration_s":0.002606,"redis_read_bytes":21,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000246,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000209524,"duration_s":0.038572,"completed_at":"2026-10-10T10:47:02.888Z","load_balancing_strategy":"primary","db_duration_s":0.014609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 145.881332] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.889Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c1710a071b8a3651dc742b3a","created_at":"2026-10-10T10:47:02.483Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9cf1be2e0894094451dcad32c8a0a66","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.494Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c1710a071b8a3651dc742b3a: start","job_status":"start","queue_duration_s":0.395495,"scheduling_latency_s":0.39551}1321gitlab # [ 145.882024] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c1710a071b8a3651dc742b3a","created_at":"2026-10-10T10:47:02.483Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9cf1be2e0894094451dcad32c8a0a66","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.494Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c1710a071b8a3651dc742b3a: done: 0.03194 sec","job_status":"done","queue_duration_s":0.395495,"scheduling_latency_s":0.39551,"redis_calls":10,"redis_duration_s":0.005357000000000001,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000411,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000242209,"duration_s":0.03194,"completed_at":"2026-10-10T10:47:02.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 145.895753] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ff705966bde15bc92105552b","created_at":"2026-10-10T10:47:02.567Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93b393ccfa35df25bd745276b8fec032","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.593Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ff705966bde15bc92105552b: start","job_status":"start","queue_duration_s":0.329358,"scheduling_latency_s":0.329372}1323gitlab # [ 145.907882] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ff705966bde15bc92105552b","created_at":"2026-10-10T10:47:02.567Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93b393ccfa35df25bd745276b8fec032","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.593Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ff705966bde15bc92105552b: done: 0.033293 sec","job_status":"done","queue_duration_s":0.329358,"scheduling_latency_s":0.329372,"redis_calls":10,"redis_duration_s":0.001639,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000297,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000206172,"duration_s":0.033293,"completed_at":"2026-10-10T10:47:02.956Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 145.964670] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.961Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"361392356cbc85c4dcb471e5","created_at":"2026-10-10T10:47:02.636Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"626e4dd36ddc7a50bcc662497a344101","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.662Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-361392356cbc85c4dcb471e5: start","job_status":"start","queue_duration_s":0.298447,"scheduling_latency_s":0.298465}1325gitlab # [ 145.965497] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"361392356cbc85c4dcb471e5","created_at":"2026-10-10T10:47:02.636Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"626e4dd36ddc7a50bcc662497a344101","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.662Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-361392356cbc85c4dcb471e5: done: 0.026683 sec","job_status":"done","queue_duration_s":0.298447,"scheduling_latency_s":0.298465,"redis_calls":12,"redis_duration_s":0.003754,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000623,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002414,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002105016,"exclusive_lock_hold_duration_s":0.006042388,"duration_s":0.026683,"completed_at":"2026-10-10T10:47:02.987Z","load_balancing_strategy":"primary","db_duration_s":0.005655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 146.006929] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:02.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d1525e6b8278e556e452a315","created_at":"2026-10-10T10:47:02.677Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9a3452a739d97b02c8d5cda29bf4dd5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T10:47:02.695Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d1525e6b8278e556e452a315: start","job_status":"start","queue_duration_s":0.303222,"scheduling_latency_s":0.303241}1327gitlab # [ 146.007693] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.010Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1328gitlab # [ 146.008192] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d1525e6b8278e556e452a315","created_at":"2026-10-10T10:47:02.677Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9a3452a739d97b02c8d5cda29bf4dd5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T10:47:02.695Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d1525e6b8278e556e452a315: done: 0.01898 sec","job_status":"done","queue_duration_s":0.303222,"scheduling_latency_s":0.303241,"redis_calls":10,"redis_duration_s":0.00196,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000942,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212597,"duration_s":0.01898,"completed_at":"2026-10-10T10:47:03.017Z","load_balancing_strategy":"primary","db_duration_s":0.00151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 146.053572] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"35891aed6ef8d46d2e7d7444","created_at":"2026-10-10T10:47:02.717Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a80c13eaafccf9f667c55158575e1a1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.734Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-35891aed6ef8d46d2e7d7444: start","job_status":"start","queue_duration_s":0.284107,"scheduling_latency_s":0.28412}1330gitlab # [ 146.264455] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"35891aed6ef8d46d2e7d7444","created_at":"2026-10-10T10:47:02.717Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a80c13eaafccf9f667c55158575e1a1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.734Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-35891aed6ef8d46d2e7d7444: done: 0.399976 sec","job_status":"done","queue_duration_s":0.284107,"scheduling_latency_s":0.28412,"redis_calls":12,"redis_duration_s":0.002429,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.129406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000345295,"exclusive_lock_hold_duration_s":0.377491274,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.399976,"completed_at":"2026-10-10T10:47:03.418Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.264663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 146.337105] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.492Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"573249a8145db115f50c836f","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-573249a8145db115f50c836f: start","job_status":"start","queue_duration_s":0.608952,"scheduling_latency_s":0.608993}1332gitlab # [ 146.363277] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.519Z","message":"No existing merge request to be cleaned up."}1333gitlab # [ 146.378059] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.533Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"573249a8145db115f50c836f","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-573249a8145db115f50c836f: done: 0.04109 sec","job_status":"done","queue_duration_s":0.608952,"scheduling_latency_s":0.608993,"redis_calls":13,"redis_duration_s":0.003956,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002671,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000328813,"duration_s":0.04109,"completed_at":"2026-10-10T10:47:03.533Z","load_balancing_strategy":"primary","db_duration_s":0.003718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 146.415942] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.571Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"991c543eb7f4e493e2669d0a","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-991c543eb7f4e493e2669d0a: start","job_status":"start","queue_duration_s":0.68791,"scheduling_latency_s":0.687945}1335gitlab # [ 146.428173] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.583Z","message":"No existing merge request to be cleaned up."}1336gitlab # [ 146.437400] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.592Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"991c543eb7f4e493e2669d0a","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-991c543eb7f4e493e2669d0a: done: 0.021609 sec","job_status":"done","queue_duration_s":0.68791,"scheduling_latency_s":0.687945,"redis_calls":10,"redis_duration_s":0.003088,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002435,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021609,"completed_at":"2026-10-10T10:47:03.592Z","load_balancing_strategy":"primary","db_duration_s":0.001535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 146.472565] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.628Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9e0c203f80b788eaf107e125","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9e0c203f80b788eaf107e125: start","job_status":"start","queue_duration_s":0.744511,"scheduling_latency_s":0.744547}1338gitlab # [ 146.492480] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.648Z","message":"No existing merge request to be cleaned up."}1339gitlab # [ 146.499707] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.655Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9e0c203f80b788eaf107e125","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9e0c203f80b788eaf107e125: done: 0.027376 sec","job_status":"done","queue_duration_s":0.744511,"scheduling_latency_s":0.744547,"redis_calls":10,"redis_duration_s":0.0030629999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002486,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027376,"completed_at":"2026-10-10T10:47:03.655Z","load_balancing_strategy":"primary","db_duration_s":0.001443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 146.534139] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.689Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"951439ddce2b3f6287c59af9","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-951439ddce2b3f6287c59af9: start","job_status":"start","queue_duration_s":0.806113,"scheduling_latency_s":0.806138}1341gitlab # [ 146.552243] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.708Z","message":"No existing merge request to be cleaned up."}1342gitlab # [ 146.559346] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.714Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"951439ddce2b3f6287c59af9","created_at":"2026-10-10T10:47:02.875Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-0b3b1678b2404d65","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f4d708db7ff8fd71c5c81ff037e2a9c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.883Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-951439ddce2b3f6287c59af9: done: 0.025151 sec","job_status":"done","queue_duration_s":0.806113,"scheduling_latency_s":0.806138,"redis_calls":10,"redis_duration_s":0.002888,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002323,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025151,"completed_at":"2026-10-10T10:47:03.714Z","load_balancing_strategy":"primary","db_duration_s":0.001333,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 146.593658] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"95aafb79ec41bf1d57a545ec","created_at":"2026-10-10T10:47:02.764Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a00ffd5e4814862ada790c9f3edc2ec9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:47:02.891Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-95aafb79ec41bf1d57a545ec: start","job_status":"start","queue_duration_s":0.857991,"scheduling_latency_s":0.858014}1344gitlab # [ 146.616452] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"95aafb79ec41bf1d57a545ec","created_at":"2026-10-10T10:47:02.764Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a00ffd5e4814862ada790c9f3edc2ec9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:47:02.891Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-95aafb79ec41bf1d57a545ec: done: 0.022822 sec","job_status":"done","queue_duration_s":0.857991,"scheduling_latency_s":0.858014,"redis_calls":10,"redis_duration_s":0.001326,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217625,"duration_s":0.022822,"completed_at":"2026-10-10T10:47:03.772Z","load_balancing_strategy":"primary","db_duration_s":0.003066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 146.653079] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"50eff0c0ad1760116aee7c4d","created_at":"2026-10-10T10:47:02.902Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.913Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-50eff0c0ad1760116aee7c4d: start","job_status":"start","queue_duration_s":0.895224,"scheduling_latency_s":0.895246}1346gitlab # [ 146.736691] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.892Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"50eff0c0ad1760116aee7c4d","created_at":"2026-10-10T10:47:02.902Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.913Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-50eff0c0ad1760116aee7c4d: done: 0.083355 sec","job_status":"done","queue_duration_s":0.895224,"scheduling_latency_s":0.895246,"redis_calls":71,"redis_duration_s":0.014169,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001522,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001046,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.01099,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000611,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000300597,"duration_s":0.083355,"completed_at":"2026-10-10T10:47:03.891Z","load_balancing_strategy":"primary","db_duration_s":0.001361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 146.800136] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"85a98ddc247db62558984f37","created_at":"2026-10-10T10:47:02.916Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10ea0bc5493e6c0140524e6cf8c71c2a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.924Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-85a98ddc247db62558984f37: start","job_status":"start","queue_duration_s":1.030745,"scheduling_latency_s":1.030791}1348gitlab # [ 146.817387] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.973Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1349gitlab # [ 146.831209] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:03.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"85a98ddc247db62558984f37","created_at":"2026-10-10T10:47:02.916Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10ea0bc5493e6c0140524e6cf8c71c2a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.924Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-85a98ddc247db62558984f37: done: 0.031185 sec","job_status":"done","queue_duration_s":1.030745,"scheduling_latency_s":1.030791,"redis_calls":10,"redis_duration_s":0.002386,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001265,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000697,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000465143,"duration_s":0.031185,"completed_at":"2026-10-10T10:47:03.986Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 146.872193] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.027Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a792d97dc885e5131d1e3a2b","created_at":"2026-10-10T10:47:02.927Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"255b8339d7b5e38c568e86a3c7bf8047","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.938Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a792d97dc885e5131d1e3a2b: start","job_status":"start","queue_duration_s":1.089064,"scheduling_latency_s":1.089106}1351gitlab # [ 147.169609] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a792d97dc885e5131d1e3a2b","created_at":"2026-10-10T10:47:02.927Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"255b8339d7b5e38c568e86a3c7bf8047","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.938Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a792d97dc885e5131d1e3a2b: done: 0.297566 sec","job_status":"done","queue_duration_s":1.089064,"scheduling_latency_s":1.089106,"redis_calls":565,"redis_duration_s":0.095993,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.090025,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.226534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029501,"duration_s":0.297566,"completed_at":"2026-10-10T10:47:04.325Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 147.226213] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8acf23afc79024834d3df0c1","created_at":"2026-10-10T10:47:02.945Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7da099df8b384a566517d84a0d62227","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:47:02.952Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8acf23afc79024834d3df0c1: start","job_status":"start","queue_duration_s":1.428725,"scheduling_latency_s":1.428753}1353gitlab # [ 147.246332] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.402Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1354gitlab # [ 147.255390] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8acf23afc79024834d3df0c1","created_at":"2026-10-10T10:47:02.945Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7da099df8b384a566517d84a0d62227","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:47:02.952Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8acf23afc79024834d3df0c1: done: 0.028774 sec","job_status":"done","queue_duration_s":1.428725,"scheduling_latency_s":1.428753,"redis_calls":10,"redis_duration_s":0.003369,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000257854,"duration_s":0.028774,"completed_at":"2026-10-10T10:47:04.410Z","load_balancing_strategy":"primary","db_duration_s":0.000689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 147.293359] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"781e9f7ca50a65ed45c9a079","created_at":"2026-10-10T10:47:02.955Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13ba9257d3990a97e2a1c5a3f0d7c045","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.970Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-781e9f7ca50a65ed45c9a079: start","job_status":"start","queue_duration_s":1.477813,"scheduling_latency_s":1.477838}1356gitlab # [ 147.340222] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"781e9f7ca50a65ed45c9a079","created_at":"2026-10-10T10:47:02.955Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13ba9257d3990a97e2a1c5a3f0d7c045","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.970Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-781e9f7ca50a65ed45c9a079: done: 0.047073 sec","job_status":"done","queue_duration_s":1.477813,"scheduling_latency_s":1.477838,"redis_calls":12,"redis_duration_s":0.002762,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001153219,"exclusive_lock_hold_duration_s":0.0049724209999999994,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.047073,"completed_at":"2026-10-10T10:47:04.495Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 147.379958] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"209f114652e45acda65242e2","created_at":"2026-10-10T10:47:02.974Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b8462eca000488c1b9f605573dfa7c0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.986Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-209f114652e45acda65242e2: start","job_status":"start","queue_duration_s":1.549086,"scheduling_latency_s":1.549127}1358gitlab # [ 147.458576] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"209f114652e45acda65242e2","created_at":"2026-10-10T10:47:02.974Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b8462eca000488c1b9f605573dfa7c0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:02.986Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-209f114652e45acda65242e2: done: 0.078271 sec","job_status":"done","queue_duration_s":1.549086,"scheduling_latency_s":1.549127,"redis_calls":12,"redis_duration_s":0.003506,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001624,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000983923,"exclusive_lock_hold_duration_s":0.018794848,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.078271,"completed_at":"2026-10-10T10:47:04.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 147.517994] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0415f456d5ae33b49e32424d","created_at":"2026-10-10T10:47:02.999Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"541dde1fce9df507228e87918c613499","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.011Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0415f456d5ae33b49e32424d: start","job_status":"start","queue_duration_s":1.661191,"scheduling_latency_s":1.661248}1360gitlab # [ 147.567701] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0415f456d5ae33b49e32424d","created_at":"2026-10-10T10:47:02.999Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-5c66a03a7590476a","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"541dde1fce9df507228e87918c613499","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.011Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0415f456d5ae33b49e32424d: done: 0.049909 sec","job_status":"done","queue_duration_s":1.661191,"scheduling_latency_s":1.661248,"redis_calls":13,"redis_duration_s":0.003038,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000714,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011327976,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.049909,"completed_at":"2026-10-10T10:47:04.722Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004246,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1361gitlab # [ 147.612514] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ae5b4af5d2c63644b27e0d1","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ae5b4af5d2c63644b27e0d1: start","job_status":"start","queue_duration_s":0.883276,"scheduling_latency_s":0.883329}1362gitlab # [ 147.656589] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ae5b4af5d2c63644b27e0d1","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ae5b4af5d2c63644b27e0d1: done: 0.043975 sec","job_status":"done","queue_duration_s":0.883276,"scheduling_latency_s":0.883329,"redis_calls":16,"redis_duration_s":0.003691,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000453409,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043975,"completed_at":"2026-10-10T10:47:04.811Z","load_balancing_strategy":"primary","db_duration_s":0.000948,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 147.694852] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3fd173efb92c64fdad8c54b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3fd173efb92c64fdad8c54b: start","job_status":"start","queue_duration_s":0.965584,"scheduling_latency_s":0.965622}1364gitlab # [ 147.724663] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3fd173efb92c64fdad8c54b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3fd173efb92c64fdad8c54b: done: 0.029815 sec","job_status":"done","queue_duration_s":0.965584,"scheduling_latency_s":0.965622,"redis_calls":13,"redis_duration_s":0.0030889999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029815,"completed_at":"2026-10-10T10:47:04.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 147.760751] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c25f1de825d28251caa7a32b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c25f1de825d28251caa7a32b: start","job_status":"start","queue_duration_s":1.031624,"scheduling_latency_s":1.03166}1366gitlab # [ 147.794197] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c25f1de825d28251caa7a32b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c25f1de825d28251caa7a32b: done: 0.03323 sec","job_status":"done","queue_duration_s":1.031624,"scheduling_latency_s":1.03166,"redis_calls":13,"redis_duration_s":0.002821,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001879,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03323,"completed_at":"2026-10-10T10:47:04.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 147.829923] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:04.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2234a79a88419ed2f183f85","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2234a79a88419ed2f183f85: start","job_status":"start","queue_duration_s":1.100726,"scheduling_latency_s":1.100768}1368gitlab # [ 147.855342] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2234a79a88419ed2f183f85","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2234a79a88419ed2f183f85: done: 0.024996 sec","job_status":"done","queue_duration_s":1.100726,"scheduling_latency_s":1.100768,"redis_calls":13,"redis_duration_s":0.002899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024996,"completed_at":"2026-10-10T10:47:05.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 147.891312] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ad603adc6c33f19401a9eae","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ad603adc6c33f19401a9eae: start","job_status":"start","queue_duration_s":1.162161,"scheduling_latency_s":1.162196}1370gitlab # [ 147.928212] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ad603adc6c33f19401a9eae","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ad603adc6c33f19401a9eae: done: 0.036397 sec","job_status":"done","queue_duration_s":1.162161,"scheduling_latency_s":1.162196,"redis_calls":13,"redis_duration_s":0.002797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036397,"completed_at":"2026-10-10T10:47:05.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 147.964049] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ff4a307172fe22d0432286c","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ff4a307172fe22d0432286c: start","job_status":"start","queue_duration_s":1.234823,"scheduling_latency_s":1.234867}1372gitlab # [ 147.998825] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ff4a307172fe22d0432286c","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ff4a307172fe22d0432286c: done: 0.034308 sec","job_status":"done","queue_duration_s":1.234823,"scheduling_latency_s":1.234867,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034308,"completed_at":"2026-10-10T10:47:05.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 148.035046] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3098dbfd3b54da75bd86474e","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3098dbfd3b54da75bd86474e: start","job_status":"start","queue_duration_s":1.305761,"scheduling_latency_s":1.305814}1374gitlab # [ 148.067915] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3098dbfd3b54da75bd86474e","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3098dbfd3b54da75bd86474e: done: 0.03253 sec","job_status":"done","queue_duration_s":1.305761,"scheduling_latency_s":1.305814,"redis_calls":13,"redis_duration_s":0.004112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002548,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03253,"completed_at":"2026-10-10T10:47:05.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 148.105051] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00cba945d93d4990551bff33","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00cba945d93d4990551bff33: start","job_status":"start","queue_duration_s":1.375796,"scheduling_latency_s":1.375837}1376gitlab # [ 148.129775] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00cba945d93d4990551bff33","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00cba945d93d4990551bff33: done: 0.024682 sec","job_status":"done","queue_duration_s":1.375796,"scheduling_latency_s":1.375837,"redis_calls":13,"redis_duration_s":0.0037879999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001365,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002423,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024682,"completed_at":"2026-10-10T10:47:05.284Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 148.151291] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93ee776b3bc3d895b16d0b4e","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93ee776b3bc3d895b16d0b4e: start","job_status":"start","queue_duration_s":1.422052,"scheduling_latency_s":1.42209}1378gitlab # [ 148.175894] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93ee776b3bc3d895b16d0b4e","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93ee776b3bc3d895b16d0b4e: done: 0.024661 sec","job_status":"done","queue_duration_s":1.422052,"scheduling_latency_s":1.42209,"redis_calls":13,"redis_duration_s":0.005042,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024661,"completed_at":"2026-10-10T10:47:05.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 148.177840] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63b1d6305eb75a25f3cf828b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63b1d6305eb75a25f3cf828b: start","job_status":"start","queue_duration_s":1.448793,"scheduling_latency_s":1.448818}1380gitlab # [ 148.189100] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63b1d6305eb75a25f3cf828b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63b1d6305eb75a25f3cf828b: done: 0.011142 sec","job_status":"done","queue_duration_s":1.448793,"scheduling_latency_s":1.448818,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011142,"completed_at":"2026-10-10T10:47:05.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 148.219473] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dc8e865a136fd330cf7d6de","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dc8e865a136fd330cf7d6de: start","job_status":"start","queue_duration_s":1.460877,"scheduling_latency_s":1.460891}1382gitlab # [ 148.230489] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dc8e865a136fd330cf7d6de","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dc8e865a136fd330cf7d6de: done: 0.009721 sec","job_status":"done","queue_duration_s":1.460877,"scheduling_latency_s":1.460891,"redis_calls":13,"redis_duration_s":0.0015149999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009721,"completed_at":"2026-10-10T10:47:05.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 148.274657] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9cef796cfbd0be1b0b3165f","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9cef796cfbd0be1b0b3165f: start","job_status":"start","queue_duration_s":1.471416,"scheduling_latency_s":1.471428}1384gitlab # [ 148.276184] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9cef796cfbd0be1b0b3165f","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9cef796cfbd0be1b0b3165f: done: 0.009854 sec","job_status":"done","queue_duration_s":1.471416,"scheduling_latency_s":1.471428,"redis_calls":13,"redis_duration_s":0.001754,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009854,"completed_at":"2026-10-10T10:47:05.365Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 148.338023] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f81254b46d19f15e8511ce3d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f81254b46d19f15e8511ce3d: start","job_status":"start","queue_duration_s":1.48209,"scheduling_latency_s":1.482102}1386gitlab # [ 148.338697] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f81254b46d19f15e8511ce3d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f81254b46d19f15e8511ce3d: done: 0.009591 sec","job_status":"done","queue_duration_s":1.48209,"scheduling_latency_s":1.482102,"redis_calls":13,"redis_duration_s":0.0015209999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009591,"completed_at":"2026-10-10T10:47:05.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 148.378749] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5986a8063fe6aeb5cd58cd9b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5986a8063fe6aeb5cd58cd9b: start","job_status":"start","queue_duration_s":1.49266,"scheduling_latency_s":1.492673}1388gitlab # [ 148.380306] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5986a8063fe6aeb5cd58cd9b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5986a8063fe6aeb5cd58cd9b: done: 0.009847 sec","job_status":"done","queue_duration_s":1.49266,"scheduling_latency_s":1.492673,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009847,"completed_at":"2026-10-10T10:47:05.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 148.432121] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26076d769eb9693d690d8db7","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26076d769eb9693d690d8db7: start","job_status":"start","queue_duration_s":1.503277,"scheduling_latency_s":1.50329}1390gitlab # [ 148.432749] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26076d769eb9693d690d8db7","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26076d769eb9693d690d8db7: done: 0.009535 sec","job_status":"done","queue_duration_s":1.503277,"scheduling_latency_s":1.50329,"redis_calls":13,"redis_duration_s":0.0015500000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009535,"completed_at":"2026-10-10T10:47:05.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 148.473783] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b33cc5cd13ff37a8d8376087","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b33cc5cd13ff37a8d8376087: start","job_status":"start","queue_duration_s":1.513789,"scheduling_latency_s":1.513802}1392gitlab # [ 148.474559] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b33cc5cd13ff37a8d8376087","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b33cc5cd13ff37a8d8376087: done: 0.01241 sec","job_status":"done","queue_duration_s":1.513789,"scheduling_latency_s":1.513802,"redis_calls":13,"redis_duration_s":0.0018319999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001287,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01241,"completed_at":"2026-10-10T10:47:05.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 148.492986] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c175cc1c9a3ce960d430213a","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c175cc1c9a3ce960d430213a: start","job_status":"start","queue_duration_s":1.527494,"scheduling_latency_s":1.527516}1394gitlab # [ 148.539021] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c175cc1c9a3ce960d430213a","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c175cc1c9a3ce960d430213a: done: 0.010602 sec","job_status":"done","queue_duration_s":1.527494,"scheduling_latency_s":1.527516,"redis_calls":13,"redis_duration_s":0.0014739999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010602,"completed_at":"2026-10-10T10:47:05.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 148.539934] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d6c8ee6c75fd92d82945ddb","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d6c8ee6c75fd92d82945ddb: start","job_status":"start","queue_duration_s":1.539026,"scheduling_latency_s":1.53904}1396gitlab # [ 148.583705] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d6c8ee6c75fd92d82945ddb","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d6c8ee6c75fd92d82945ddb: done: 0.014853 sec","job_status":"done","queue_duration_s":1.539026,"scheduling_latency_s":1.53904,"redis_calls":13,"redis_duration_s":0.006463,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014853,"completed_at":"2026-10-10T10:47:05.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 148.584717] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"814720a5557767428ae6fd82","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-814720a5557767428ae6fd82: start","job_status":"start","queue_duration_s":1.556503,"scheduling_latency_s":1.556519}1398gitlab # [ 148.632058] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"814720a5557767428ae6fd82","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-814720a5557767428ae6fd82: done: 0.017231 sec","job_status":"done","queue_duration_s":1.556503,"scheduling_latency_s":1.556519,"redis_calls":13,"redis_duration_s":0.007518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017231,"completed_at":"2026-10-10T10:47:05.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 148.633065] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e58fde6a9242d490589341b4","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e58fde6a9242d490589341b4: start","job_status":"start","queue_duration_s":1.574958,"scheduling_latency_s":1.574972}1400gitlab # [ 148.641024] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e58fde6a9242d490589341b4","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e58fde6a9242d490589341b4: done: 0.010258 sec","job_status":"done","queue_duration_s":1.574958,"scheduling_latency_s":1.574972,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010258,"completed_at":"2026-10-10T10:47:05.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 148.700597] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a61b9c7c8072e7191f7d75","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a61b9c7c8072e7191f7d75: start","job_status":"start","queue_duration_s":1.586069,"scheduling_latency_s":1.586084}1402gitlab # [ 148.701562] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a61b9c7c8072e7191f7d75","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a61b9c7c8072e7191f7d75: done: 0.010061 sec","job_status":"done","queue_duration_s":1.586069,"scheduling_latency_s":1.586084,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010061,"completed_at":"2026-10-10T10:47:05.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 148.721936] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f926550b6542f332f4b0082a","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f926550b6542f332f4b0082a: start","job_status":"start","queue_duration_s":1.597017,"scheduling_latency_s":1.59703}1404gitlab # [ 148.733217] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f926550b6542f332f4b0082a","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f926550b6542f332f4b0082a: done: 0.010132 sec","job_status":"done","queue_duration_s":1.597017,"scheduling_latency_s":1.59703,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010132,"completed_at":"2026-10-10T10:47:05.491Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 148.779742] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9369c00c94ad529c89273cfb","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9369c00c94ad529c89273cfb: start","job_status":"start","queue_duration_s":1.608061,"scheduling_latency_s":1.608075}1406gitlab # [ 148.780480] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9369c00c94ad529c89273cfb","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9369c00c94ad529c89273cfb: done: 0.012272 sec","job_status":"done","queue_duration_s":1.608061,"scheduling_latency_s":1.608075,"redis_calls":13,"redis_duration_s":0.0015829999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012272,"completed_at":"2026-10-10T10:47:05.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 148.843748] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5ed042ff27ee648cc75e7de","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5ed042ff27ee648cc75e7de: start","job_status":"start","queue_duration_s":1.62138,"scheduling_latency_s":1.621395}1408gitlab # [ 148.844619] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5ed042ff27ee648cc75e7de","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5ed042ff27ee648cc75e7de: done: 0.010224 sec","job_status":"done","queue_duration_s":1.62138,"scheduling_latency_s":1.621395,"redis_calls":13,"redis_duration_s":0.00154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010224,"completed_at":"2026-10-10T10:47:05.516Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 148.890759] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"829265a0f9d48a4462e8d4a1","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-829265a0f9d48a4462e8d4a1: start","job_status":"start","queue_duration_s":1.632498,"scheduling_latency_s":1.632512}1410gitlab # [ 148.891640] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"829265a0f9d48a4462e8d4a1","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-829265a0f9d48a4462e8d4a1: done: 0.010208 sec","job_status":"done","queue_duration_s":1.632498,"scheduling_latency_s":1.632512,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010208,"completed_at":"2026-10-10T10:47:05.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 148.931731] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9edba6c74eee622246a8691e","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9edba6c74eee622246a8691e: start","job_status":"start","queue_duration_s":1.643568,"scheduling_latency_s":1.643581}1412gitlab # [ 148.932671] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9edba6c74eee622246a8691e","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9edba6c74eee622246a8691e: done: 0.015104 sec","job_status":"done","queue_duration_s":1.643568,"scheduling_latency_s":1.643581,"redis_calls":13,"redis_duration_s":0.0066489999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002142,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004507,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015104,"completed_at":"2026-10-10T10:47:05.543Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 148.983921] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a25bd3cafbd6fce60e881f47","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a25bd3cafbd6fce60e881f47: start","job_status":"start","queue_duration_s":1.661612,"scheduling_latency_s":1.661626}1414gitlab # [ 148.984712] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a25bd3cafbd6fce60e881f47","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a25bd3cafbd6fce60e881f47: done: 0.010975 sec","job_status":"done","queue_duration_s":1.661612,"scheduling_latency_s":1.661626,"redis_calls":13,"redis_duration_s":0.0015470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010975,"completed_at":"2026-10-10T10:47:05.557Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 149.023709] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a570fcaa5b4e8fda24fa6152","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a570fcaa5b4e8fda24fa6152: start","job_status":"start","queue_duration_s":1.673583,"scheduling_latency_s":1.673597}1416gitlab # [ 149.024604] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a570fcaa5b4e8fda24fa6152","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a570fcaa5b4e8fda24fa6152: done: 0.01005 sec","job_status":"done","queue_duration_s":1.673583,"scheduling_latency_s":1.673597,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01005,"completed_at":"2026-10-10T10:47:05.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 149.075707] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"450a5acf8b3ac323074ec914","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-450a5acf8b3ac323074ec914: start","job_status":"start","queue_duration_s":1.684627,"scheduling_latency_s":1.68464}1418gitlab # [ 149.076595] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"450a5acf8b3ac323074ec914","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-450a5acf8b3ac323074ec914: done: 0.009919 sec","job_status":"done","queue_duration_s":1.684627,"scheduling_latency_s":1.68464,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009919,"completed_at":"2026-10-10T10:47:05.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 149.088663] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24bdffcaa690af766542b889","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24bdffcaa690af766542b889: start","job_status":"start","queue_duration_s":1.695552,"scheduling_latency_s":1.695566}1420gitlab # [ 149.137047] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24bdffcaa690af766542b889","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24bdffcaa690af766542b889: done: 0.013213 sec","job_status":"done","queue_duration_s":1.695552,"scheduling_latency_s":1.695566,"redis_calls":13,"redis_duration_s":0.004405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001427,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013213,"completed_at":"2026-10-10T10:47:05.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 149.137560] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21465a68bdb045ab2885e999","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21465a68bdb045ab2885e999: start","job_status":"start","queue_duration_s":1.7098,"scheduling_latency_s":1.709814}1422gitlab # [ 149.145154] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21465a68bdb045ab2885e999","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21465a68bdb045ab2885e999: done: 0.010149 sec","job_status":"done","queue_duration_s":1.7098,"scheduling_latency_s":1.709814,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010149,"completed_at":"2026-10-10T10:47:05.604Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 149.185111] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"135160cd37b6ceab3254fe25","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-135160cd37b6ceab3254fe25: start","job_status":"start","queue_duration_s":1.720784,"scheduling_latency_s":1.7208}1424gitlab # [ 149.190688] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"135160cd37b6ceab3254fe25","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-135160cd37b6ceab3254fe25: done: 0.010135 sec","job_status":"done","queue_duration_s":1.720784,"scheduling_latency_s":1.7208,"redis_calls":13,"redis_duration_s":0.0016619999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010135,"completed_at":"2026-10-10T10:47:05.615Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 149.230316] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7e0b2cc85c54b5474a52608","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7e0b2cc85c54b5474a52608: start","job_status":"start","queue_duration_s":1.73175,"scheduling_latency_s":1.731766}1426gitlab # [ 149.236192] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7e0b2cc85c54b5474a52608","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7e0b2cc85c54b5474a52608: done: 0.010278 sec","job_status":"done","queue_duration_s":1.73175,"scheduling_latency_s":1.731766,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010278,"completed_at":"2026-10-10T10:47:05.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 149.280235] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55b14112d38fcca0b87b3873","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55b14112d38fcca0b87b3873: start","job_status":"start","queue_duration_s":1.742872,"scheduling_latency_s":1.742886}1428gitlab # [ 149.281741] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55b14112d38fcca0b87b3873","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55b14112d38fcca0b87b3873: done: 0.020332 sec","job_status":"done","queue_duration_s":1.742872,"scheduling_latency_s":1.742886,"redis_calls":13,"redis_duration_s":0.011592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002368,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.009224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020332,"completed_at":"2026-10-10T10:47:05.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 149.347242] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c978a627eb99481ea762384d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c978a627eb99481ea762384d: start","job_status":"start","queue_duration_s":1.765642,"scheduling_latency_s":1.765657}1430gitlab # [ 149.347880] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c978a627eb99481ea762384d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c978a627eb99481ea762384d: done: 0.012619 sec","job_status":"done","queue_duration_s":1.765642,"scheduling_latency_s":1.765657,"redis_calls":13,"redis_duration_s":0.0031320000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001621,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012619,"completed_at":"2026-10-10T10:47:05.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1431gitlab # [ 149.363763] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e4edf575e84b0f088c88a60","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e4edf575e84b0f088c88a60: start","job_status":"start","queue_duration_s":1.77913,"scheduling_latency_s":1.779144}1432gitlab # [ 149.375090] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e4edf575e84b0f088c88a60","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e4edf575e84b0f088c88a60: done: 0.01009 sec","job_status":"done","queue_duration_s":1.77913,"scheduling_latency_s":1.779144,"redis_calls":13,"redis_duration_s":0.001712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01009,"completed_at":"2026-10-10T10:47:05.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 149.428732] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dba7a3f22b822ced8fa71e53","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dba7a3f22b822ced8fa71e53: start","job_status":"start","queue_duration_s":1.790378,"scheduling_latency_s":1.790391}1434gitlab # [ 149.429678] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dba7a3f22b822ced8fa71e53","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dba7a3f22b822ced8fa71e53: done: 0.009916 sec","job_status":"done","queue_duration_s":1.790378,"scheduling_latency_s":1.790391,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009916,"completed_at":"2026-10-10T10:47:05.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 149.456469] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a99328de9442cdb7ebdb5e74","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a99328de9442cdb7ebdb5e74: start","job_status":"start","queue_duration_s":1.801292,"scheduling_latency_s":1.801332}1436gitlab # [ 149.467696] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a99328de9442cdb7ebdb5e74","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a99328de9442cdb7ebdb5e74: done: 0.010098 sec","job_status":"done","queue_duration_s":1.801292,"scheduling_latency_s":1.801332,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010098,"completed_at":"2026-10-10T10:47:05.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 149.509957] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc6e2c64d6d87dc63637c4b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc6e2c64d6d87dc63637c4b: start","job_status":"start","queue_duration_s":1.814656,"scheduling_latency_s":1.814671}1438gitlab # [ 149.513088] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc6e2c64d6d87dc63637c4b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc6e2c64d6d87dc63637c4b: done: 0.010059 sec","job_status":"done","queue_duration_s":1.814656,"scheduling_latency_s":1.814671,"redis_calls":13,"redis_duration_s":0.0015339999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010059,"completed_at":"2026-10-10T10:47:05.709Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 149.555306] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e72aed061bb6602a566dd950","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e72aed061bb6602a566dd950: start","job_status":"start","queue_duration_s":1.825717,"scheduling_latency_s":1.82573}1440gitlab # [ 149.594762] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e72aed061bb6602a566dd950","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e72aed061bb6602a566dd950: done: 0.010178 sec","job_status":"done","queue_duration_s":1.825717,"scheduling_latency_s":1.82573,"redis_calls":13,"redis_duration_s":0.001502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010178,"completed_at":"2026-10-10T10:47:05.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 149.595877] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4389cabb16ae0601a7ce5155","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4389cabb16ae0601a7ce5155: start","job_status":"start","queue_duration_s":1.837021,"scheduling_latency_s":1.837036}1442gitlab # [ 149.604089] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4389cabb16ae0601a7ce5155","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4389cabb16ae0601a7ce5155: done: 0.012857 sec","job_status":"done","queue_duration_s":1.837021,"scheduling_latency_s":1.837036,"redis_calls":13,"redis_duration_s":0.0018999999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012857,"completed_at":"2026-10-10T10:47:05.734Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 149.644282] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02b2d0ba96224f9b85d830db","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02b2d0ba96224f9b85d830db: start","job_status":"start","queue_duration_s":1.851084,"scheduling_latency_s":1.851108}1444gitlab # [ 149.649717] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02b2d0ba96224f9b85d830db","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02b2d0ba96224f9b85d830db: done: 0.018675 sec","job_status":"done","queue_duration_s":1.851084,"scheduling_latency_s":1.851108,"redis_calls":13,"redis_duration_s":0.008814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006662,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018675,"completed_at":"2026-10-10T10:47:05.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 149.691101] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cacbc29c8ece7569a07dbd9d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cacbc29c8ece7569a07dbd9d: start","job_status":"start","queue_duration_s":1.870653,"scheduling_latency_s":1.870668}1446gitlab # [ 149.695181] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cacbc29c8ece7569a07dbd9d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cacbc29c8ece7569a07dbd9d: done: 0.010782 sec","job_status":"done","queue_duration_s":1.870653,"scheduling_latency_s":1.870668,"redis_calls":13,"redis_duration_s":0.001712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010782,"completed_at":"2026-10-10T10:47:05.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 149.741081] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"172c5279fa43920c4ab8948a","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-172c5279fa43920c4ab8948a: start","job_status":"start","queue_duration_s":1.882258,"scheduling_latency_s":1.882272}1448gitlab # [ 149.741804] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"172c5279fa43920c4ab8948a","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-172c5279fa43920c4ab8948a: done: 0.010602 sec","job_status":"done","queue_duration_s":1.882258,"scheduling_latency_s":1.882272,"redis_calls":13,"redis_duration_s":0.001575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010602,"completed_at":"2026-10-10T10:47:05.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 149.792597] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"060eb161aa15b50c66b1e9c3","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-060eb161aa15b50c66b1e9c3: start","job_status":"start","queue_duration_s":1.893744,"scheduling_latency_s":1.893759}1450gitlab # [ 149.792765] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"060eb161aa15b50c66b1e9c3","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-060eb161aa15b50c66b1e9c3: done: 0.010164 sec","job_status":"done","queue_duration_s":1.893744,"scheduling_latency_s":1.893759,"redis_calls":13,"redis_duration_s":0.001547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010164,"completed_at":"2026-10-10T10:47:05.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 149.837081] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a825f45d2918ad9497cf6e41","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a825f45d2918ad9497cf6e41: start","job_status":"start","queue_duration_s":1.904905,"scheduling_latency_s":1.904919}1452gitlab # [ 149.837805] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a825f45d2918ad9497cf6e41","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a825f45d2918ad9497cf6e41: done: 0.010511 sec","job_status":"done","queue_duration_s":1.904905,"scheduling_latency_s":1.904919,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010511,"completed_at":"2026-10-10T10:47:05.799Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 149.867075] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c08b1689dcd2e7630c6af0ba","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c08b1689dcd2e7630c6af0ba: start","job_status":"start","queue_duration_s":1.91649,"scheduling_latency_s":1.916505}1454gitlab # [ 149.911824] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c08b1689dcd2e7630c6af0ba","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c08b1689dcd2e7630c6af0ba: done: 0.010455 sec","job_status":"done","queue_duration_s":1.91649,"scheduling_latency_s":1.916505,"redis_calls":13,"redis_duration_s":0.001766,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010455,"completed_at":"2026-10-10T10:47:05.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 149.912852] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a753b480da8d93e30f43f9af","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a753b480da8d93e30f43f9af: start","job_status":"start","queue_duration_s":1.927835,"scheduling_latency_s":1.927849}1456gitlab # [ 149.923558] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a753b480da8d93e30f43f9af","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a753b480da8d93e30f43f9af: done: 0.009851 sec","job_status":"done","queue_duration_s":1.927835,"scheduling_latency_s":1.927849,"redis_calls":13,"redis_duration_s":0.001661,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009851,"completed_at":"2026-10-10T10:47:05.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 149.965259] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7f2fe188cf2d9b5c5c8ed2b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7f2fe188cf2d9b5c5c8ed2b: start","job_status":"start","queue_duration_s":1.938583,"scheduling_latency_s":1.938597}1458gitlab # [ 150.006324] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7f2fe188cf2d9b5c5c8ed2b","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7f2fe188cf2d9b5c5c8ed2b: done: 0.009802 sec","job_status":"done","queue_duration_s":1.938583,"scheduling_latency_s":1.938597,"redis_calls":13,"redis_duration_s":0.001621,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009802,"completed_at":"2026-10-10T10:47:05.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 150.007443] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f459dbb5ec9417f088a9fe7d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f459dbb5ec9417f088a9fe7d: start","job_status":"start","queue_duration_s":1.949522,"scheduling_latency_s":1.949536}1460gitlab # [ 150.014671] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:05.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f459dbb5ec9417f088a9fe7d","created_at":"2026-10-10T10:47:03.833Z","trace_propagation_headers":{"sentry-trace":"9b5bed0d2ee7495284ff545ca0906254-59e11f6356d74c52","baggage":"sentry-trace_id=9b5bed0d2ee7495284ff545ca0906254,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8a3f9c74eef0f75743c5e982b4119e8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:03.884Z","job_size_bytes":2,"pid":1635,"sidekiq_tid":"maev","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f459dbb5ec9417f088a9fe7d: done: 0.009644 sec","job_status":"done","queue_duration_s":1.949522,"scheduling_latency_s":1.949536,"redis_calls":13,"redis_duration_s":0.001599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009644,"completed_at":"2026-10-10T10:47:05.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 150.234722] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1816085103 wd_nsec: 18160849291462gitlab # [ 163.833892] setup-gitlab-start[2232]: W, [2026-10-10T10:47:20.988840 #2232] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1463gitlab # [ 186.066136] postgres[2437]: [2437] ERROR: relation "table_without_models" does not exist1464gitlab # [ 186.067633] postgres[2437]: [2437] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1465gitlab # [ 188.979636] runuser[2231]: pam_unix(runuser:session): session closed for user gitlab1466gitlab # [ 188.983128] setup-gitlab-start[2212]: + systemctl restart gitlab-sidekiq.service1467gitlab # [ 189.008985] systemd[1]: Stopping gitlab-sidekiq.service...1468gitlab # [ 189.010096] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:46.164Z","message":"Shutting down"}1469gitlab # [ 189.011092] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:46.165Z","message":"Scheduler exiting..."}1470gitlab # [ 189.011774] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:46.166Z","message":"Terminating quiet threads for default capsule"}1471gitlab # [ 189.012864] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:46.166Z","message":"Scheduler exiting..."}1472gitlab # [ 189.514854] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:46.670Z","message":"Bye!"}1473gitlab # [ 189.539389] sidekiq-cluster[1635]: {"severity":"INFO","time":"2026-10-10T10:47:46.694Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1635,"worker_id":"sidekiq_0","memwd_rss_bytes":959037440,"retry":0}1474gitlab # [ 193.770887] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1475gitlab # [ 193.772206] systemd[1]: Stopped gitlab-sidekiq.service.1476gitlab # [ 193.773931] systemd[1]: gitlab-sidekiq.service: Consumed 58.555s CPU time over 1min 58.265s wall clock time, 973.4M memory peak, 4.4M read from disk.1477gitlab # [ 193.776505] systemd[1]: Started gitlab-sidekiq.service.1478gitlab # [ 193.781695] setup-gitlab-start[2212]: + mkdir -p /var/lib/secrets1479gitlab # [ 193.783934] setup-gitlab-start[2212]: + echo nixbot-test-token-of-20-chars1480gitlab # [ 193.785417] setup-gitlab-start[2212]: + chmod 644 /var/lib/secrets/gitlab-token1481gitlab # [ 193.791797] setup-gitlab-start[2212]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1482gitlab # [ 194.140039] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:51 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 3181483gitlab # [ 194.699905] gitaly[892]: time="2026-10-10T10:47:51.854Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:01.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:51.854" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1484gitlab # [ 194.701112] gitaly[892]: time="2026-10-10T10:47:51.854Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:01.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:51.854" grpc.time_ms=0.265 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1485gitlab # [ 194.861877] gitaly[892]: time="2026-10-10T10:47:52.017Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.017" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1486gitlab # [ 194.863106] gitaly[892]: time="2026-10-10T10:47:52.017Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.017" grpc.time_ms=0.4 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1487gitlab # [ 194.944597] gitaly[892]: time="2026-10-10T10:47:52.100Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ diskcache=1745b7e9-4ebe-41c6-a97f-a25f5a412ea4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:48:02.036" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.046" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1488gitlab # [ 194.947578] gitaly[892]: time="2026-10-10T10:47:52.100Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1579 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:48:02.036" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.046" grpc.time_ms=53.638 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 194.961738] gitaly[892]: time="2026-10-10T10:47:52.117Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.106" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.116" pid=892 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1490gitlab # [ 194.962843] gitaly[892]: time="2026-10-10T10:47:52.117Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.106" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.116" grpc.time_ms=0.798 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1491gitlab # [ 194.980237] gitaly[892]: time="2026-10-10T10:47:52.135Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:02.114" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.124" grpc.time_ms=11.091 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 195.035845] gitaly[892]: time="2026-10-10T10:47:52.191Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.180" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.191" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1493gitlab # [ 195.037585] gitaly[892]: time="2026-10-10T10:47:52.191Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.180" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.191" grpc.time_ms=0.376 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1494gitlab # [ 195.079201] gitaly[892]: time="2026-10-10T10:47:52.234Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ diskcache=ac8fc089-d843-45ff-bb3c-f2d022c17fe5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:48:02.183" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.193" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1495gitlab # [ 195.079737] gitaly[892]: time="2026-10-10T10:47:52.235Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1569 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:48:02.183" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.193" grpc.time_ms=41.298 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1496gitlab # [ 195.085636] gitaly[892]: time="2026-10-10T10:47:52.241Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.240" pid=892 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1497gitlab # [ 195.086487] gitaly[892]: time="2026-10-10T10:47:52.241Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.240" grpc.time_ms=0.736 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1498gitlab # [ 195.094918] gitaly[892]: time="2026-10-10T10:47:52.250Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:02.234" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.245" grpc.time_ms=5.251 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1499gitlab # [ 195.117434] gitaly[892]: time="2026-10-10T10:47:52.273Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:02.262" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.272" grpc.time_ms=0.274 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1500gitlab # [ 195.122451] gitaly[892]: time="2026-10-10T10:47:52.278Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:02.267" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.277" grpc.time_ms=0.372 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1501gitlab # [ 195.692072] gitaly[892]: time="2026-10-10T10:47:52.847Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT88KNPECK2PDGZ1J2AXQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:02.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:52.836" grpc.time_ms=10.879 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1502gitlab # [ 195.951643] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:53 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 17901503gitlab # [ 195.956267] setup-gitlab-start[2493]: {"id":1,"message":null,"error":null}1504gitlab # [ 196.894823] sidekiq-cluster[2480]: {"severity":"INFO","time":"2026-10-10T10:47:54.033Z","message":"Starting cluster with 1 processes"}1505gitlab # [ 201.246552] gitaly[892]: time="2026-10-10T10:47:58.401Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTEXPJYR336EA8QC3G1FJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:08.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:58.392" grpc.time_ms=8.868 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1506gitlab # [ 201.766846] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:58 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 7861507gitlab # [ 201.773351] setup-gitlab-start[2212]: + mkdir -p /tmp/test-flake1508gitlab # [ 201.776493] setup-gitlab-start[2212]: + cd /tmp/test-flake1509gitlab # [ 201.777539] setup-gitlab-start[2212]: + git init -b master1510gitlab # [ 201.807402] setup-gitlab-start[2579]: Initialized empty Git repository in /tmp/test-flake/.git/1511gitlab # [ 201.809930] setup-gitlab-start[2212]: + git config user.name test1512gitlab # [ 201.815842] setup-gitlab-start[2212]: + git config user.email test@example.com1513gitlab # [ 201.821878] setup-gitlab-start[2212]: + cat1514gitlab # [ 201.827313] setup-gitlab-start[2212]: + git add flake.nix1515gitlab # [ 201.832687] setup-gitlab-start[2212]: + git commit -m 'initial commit'1516gitlab # [ 201.860926] setup-gitlab-start[2584]: [master (root-commit) f294d2c] initial commit1517gitlab # [ 201.862768] setup-gitlab-start[2584]: 1 file changed, 10 insertions(+)1518gitlab # [ 201.864127] setup-gitlab-start[2584]: create mode 100644 flake.nix1519gitlab # [ 201.865332] setup-gitlab-start[2212]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1520gitlab # [ 201.869947] setup-gitlab-start[2212]: + git push -u origin master1521gitlab # [ 201.940319] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:59 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 381522gitlab # [ 202.048607] gitaly[892]: time="2026-10-10T10:47:59.204Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTFVX3HR8EVM96WHVE8CW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:47:59.193" grpc.time_ms=10.751 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1523gitlab # [ 202.050322] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:59 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1031524gitlab # [ 202.290594] gitaly[892]: time="2026-10-10T10:47:59.445Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:09.043" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:59.434" grpc.time_ms=11.072 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1525gitlab # [ 202.359782] gitaly[892]: time="2026-10-10T10:47:59.515Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-10T10:48:29.003" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-10T10:47:59.503" grpc.time_ms=11.738 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1526gitlab # [ 202.419891] gitaly[892]: time="2026-10-10T10:47:59.575Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4JPTG0HGYX54QCC73SYRZHH duration_ms=200ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/allowed"1527gitlab # [ 202.451628] gitaly[892]: time="2026-10-10T10:47:59.607Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4JPTG0HGYX54QCC73SYRZHH duration_ms=30ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/pre_receive"1528gitlab # [ 202.452749] gitaly[892]: time="2026-10-10T10:47:59.608Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:59.372" grpc.time_ms=234.911 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1529gitlab # [ 202.514801] gitaly[892]: time="2026-10-10T10:47:59.670Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:59.669" grpc.time_ms=0.788 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1530gitlab # [ 202.563961] gitaly[892]: time="2026-10-10T10:47:59.719Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:59.718" grpc.time_ms=1.086 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1531gitlab # [ 202.601951] gitaly[892]: time="2026-10-10T10:47:59.757Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:59.757" grpc.time_ms=0.35 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1532gitlab # [ 202.721774] gitaly[892]: time="2026-10-10T10:47:59.877Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:09.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:59.866" grpc.time_ms=10.645 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1533gitlab # [ 202.741577] gitaly[892]: time="2026-10-10T10:47:59.897Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-10T10:48:09.022" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:47:59.883" grpc.time_ms=13.564 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1534gitlab # [ 202.750669] gitaly[892]: time="2026-10-10T10:47:59.906Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4JPTG0HGYX54QCC73SYRZHH duration_ms=118ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/post_receive"1535gitlab # [ 202.751875] gitaly[892]: time="2026-10-10T10:47:59.907Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:59.787" grpc.time_ms=119.707 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1536gitlab # [ 202.795690] gitaly[892]: time="2026-10-10T10:47:59.951Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH diskcache=3a306f7f-6f78-4178-8494-be8f844bdfeb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:47:59.303" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1537gitlab # [ 202.805128] gitaly[892]: time="2026-10-10T10:47:59.951Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=194 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13617 command.oublock=32 command.real_time_ms=639 command.spawn_token_fork_ms=0 command.system_time_ms=117 command.user_time_ms=77 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:47:59.303" grpc.time_ms=648.358 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1538gitlab # [ 202.818624] gitlab-workhorse[828]: time="2026-10-10T10:47:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521539gitlab # [ 202.819575] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:59 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7031540gitlab # [ 202.820792] setup-gitlab-start[2588]: To http://gitlab/root/test-flake.git1541gitlab # [ 202.821720] setup-gitlab-start[2588]: * [new branch] master -> master1542gitlab # [ 202.822571] setup-gitlab-start[2588]: branch 'master' set up to track 'origin/master'.1543gitlab # [ 202.827347] systemd[1]: Finished setup-gitlab.service.1544gitlab # [ 202.830778] systemd[1]: Starting nixbot CI service...1545gitlab: (finished: waiting for unit setup-gitlab.service, in 204.26 seconds)1546gitlab: waiting for unit nixbot.service1547gitlab # [ 204.082822] nixbot[2666]: {"ts": "2026-10-10T10:48:01.237Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1548gitlab # [ 204.124805] nixbot[2666]: {"ts": "2026-10-10T10:48:01.280Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1549gitlab # [ 204.179602] nixbot[2666]: {"ts": "2026-10-10T10:48:01.334Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1550gitlab # [ 204.207646] nixbot[2666]: {"ts": "2026-10-10T10:48:01.363Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1551gitlab # [ 204.230982] nixbot[2666]: {"ts": "2026-10-10T10:48:01.386Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1552gitlab # [ 204.254675] nixbot[2666]: {"ts": "2026-10-10T10:48:01.409Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1553gitlab # [ 204.269071] nixbot[2666]: {"ts": "2026-10-10T10:48:01.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1554gitlab # [ 204.281099] nixbot[2666]: {"ts": "2026-10-10T10:48:01.436Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1555gitlab # [ 204.298111] nixbot[2666]: {"ts": "2026-10-10T10:48:01.453Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1556gitlab # [ 204.306401] nixbot[2666]: {"ts": "2026-10-10T10:48:01.462Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1557gitlab # [ 204.320180] nixbot[2666]: {"ts": "2026-10-10T10:48:01.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1558gitlab # [ 204.334649] nixbot[2666]: {"ts": "2026-10-10T10:48:01.489Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1559gitlab # [ 204.345263] nixbot[2666]: {"ts": "2026-10-10T10:48:01.500Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1560gitlab # [ 204.353912] nixbot[2666]: {"ts": "2026-10-10T10:48:01.509Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1561gitlab # [ 204.365075] nixbot[2666]: {"ts": "2026-10-10T10:48:01.520Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1562gitlab # [ 204.373423] nixbot[2666]: {"ts": "2026-10-10T10:48:01.529Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1563gitlab # [ 204.381518] nixbot[2666]: {"ts": "2026-10-10T10:48:01.537Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1564gitlab # [ 204.388858] nixbot[2666]: {"ts": "2026-10-10T10:48:01.544Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1565gitlab # [ 204.400378] nixbot[2666]: {"ts": "2026-10-10T10:48:01.555Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1566gitlab # [ 204.408437] nixbot[2666]: {"ts": "2026-10-10T10:48:01.564Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1567gitlab # [ 204.416401] nixbot[2666]: {"ts": "2026-10-10T10:48:01.572Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1568gitlab # [ 204.428231] nixbot[2666]: {"ts": "2026-10-10T10:48:01.583Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1569gitlab # [ 204.436627] nixbot[2666]: {"ts": "2026-10-10T10:48:01.592Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1570gitlab # [ 204.445078] nixbot[2666]: {"ts": "2026-10-10T10:48:01.600Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1571gitlab # [ 204.452762] nixbot[2666]: {"ts": "2026-10-10T10:48:01.608Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1572gitlab # [ 204.461604] nixbot[2666]: {"ts": "2026-10-10T10:48:01.617Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1573gitlab # [ 204.469988] nixbot[2666]: {"ts": "2026-10-10T10:48:01.625Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1574gitlab # [ 204.477160] nixbot[2666]: {"ts": "2026-10-10T10:48:01.632Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1575gitlab # [ 204.484891] nixbot[2666]: {"ts": "2026-10-10T10:48:01.640Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1576gitlab # [ 204.507414] nixbot[2666]: {"ts": "2026-10-10T10:48:01.662Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1577gitlab # [ 204.518714] nixbot[2666]: {"ts": "2026-10-10T10:48:01.674Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1578gitlab # [ 204.527334] nixbot[2666]: {"ts": "2026-10-10T10:48:01.682Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1579gitlab # [ 204.534593] nixbot[2666]: {"ts": "2026-10-10T10:48:01.690Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1580gitlab # [ 204.546173] nixbot[2666]: {"ts": "2026-10-10T10:48:01.701Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1581gitlab # [ 204.549548] nixbot[2666]: {"ts": "2026-10-10T10:48:01.705Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1582gitlab # [ 204.558439] nixbot[2666]: {"ts": "2026-10-10T10:48:01.714Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1583gitlab # [ 204.568203] nixbot[2666]: {"ts": "2026-10-10T10:48:01.723Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1584gitlab # [ 204.578854] nixbot[2666]: {"ts": "2026-10-10T10:48:01.734Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1585gitlab # [ 204.950528] nixbot[2666]: {"ts": "2026-10-10T10:48:02.106Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2666]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1586gitlab # [ 204.956554] nixbot[2666]: {"ts": "2026-10-10T10:48:02.111Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1587gitlab # [ 204.993342] nixbot[2666]: {"ts": "2026-10-10T10:48:02.149Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43654 - \"GET /health HTTP/1.1\" 200"}1588gitlab # [ 205.000557] systemd[1]: Started nixbot CI service.1589gitlab # [ 205.001636] systemd[1]: Reached target Multi-User System.1590gitlab # [ 205.002730] systemd[1]: Startup finished in 1.175s (kernel) + 3.681s (initrd) + 3min 20.145s (userspace) = 3min 25.002s.1591gitlab # [ 205.275735] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2821592gitlab # [ 205.297628] nixbot[2666]: {"ts": "2026-10-10T10:48:02.452Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1593gitlab # [ 205.352920] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 411594gitlab # [ 205.357889] nixbot[2666]: {"ts": "2026-10-10T10:48:02.510Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1595gitlab # [ 205.461434] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1051596gitlab # [ 205.569436] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 981597gitlab # [ 205.665420] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 791598gitlab # [ 205.723779] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 461599gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1600gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1601gitlab # [ 205.781692] nixbot[2666]: {"ts": "2026-10-10T10:48:02.937Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43658 - \"GET /health HTTP/1.1\" 200"}1602gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1603subtest: project discovered and webhook registered1604gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1605gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds)1606(finished: subtest: project discovered and webhook registered, in 0.08 seconds)1607subtest: push delivers the webhook1608gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1609gitlab # [ 205.871311] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 511610gitlab # [ 205.879515] gitaly[892]: time="2026-10-10T10:48:03.034Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTKJ5ECPJ9RKAMXFM194H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-10T10:48:33.023" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:03.023" grpc.time_ms=10.716 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1611gitlab # [ 205.894533] gitaly[892]: time="2026-10-10T10:48:03.050Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTKJ5ECPJ9RKAMXFM194H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:33.048" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:03.049" grpc.time_ms=0.491 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1612gitlab # [ 205.905834] gitaly[892]: time="2026-10-10T10:48:03.061Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTKJ5ECPJ9RKAMXFM194H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-10T10:48:13.044" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:03.055" grpc.time_ms=6.508 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1613gitlab # [ 205.928898] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1981614gitlab # [ 205.997076] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 611615gitlab # [ 206.008608] nixbot[2666]: {"ts": "2026-10-10T10:48:03.164Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1616gitlab # [ 206.148206] gitaly[892]: time="2026-10-10T10:48:03.303Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTKXE1YF7PY16VW44DJFX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.289" grpc.time_ms=13.927 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1617gitlab # [ 206.177240] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 851618gitlab # [ 206.304086] gitaly[892]: time="2026-10-10T10:48:03.459Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTM074CR15PVK2ABZ54HP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.446" pid=892 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1619gitlab # [ 206.306183] gitaly[892]: time="2026-10-10T10:48:03.459Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTM074CR15PVK2ABZ54HP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.446" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1620gitlab # [ 206.308214] gitaly[892]: time="2026-10-10T10:48:03.463Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTM074CR15PVK2ABZ54HP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.446" grpc.time_ms=16.778 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1621gitlab # [ 206.310809] gitlab-workhorse[828]: time="2026-10-10T10:48:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131622gitlab # [ 206.313252] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1291623gitlab # [ 206.378314] gitaly[892]: time="2026-10-10T10:48:03.534Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTM4E4F2KW6HQNAYVSY1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.521" pid=892 remote_ip="2001:db8:1::1" request_sha=953c134b3afafe0b45dcc784e9d46117db02817d response_bytes=223 span.kind=server system=grpc1624gitlab # [ 206.387821] gitaly[892]: time="2026-10-10T10:48:03.543Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTM4E4F2KW6HQNAYVSY1Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.521" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1625gitlab # [ 206.388718] gitaly[892]: time="2026-10-10T10:48:03.543Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTM4E4F2KW6HQNAYVSY1Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.521" grpc.time_ms=21.787 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1626gitlab # [ 206.413570] gitlab-workhorse[828]: time="2026-10-10T10:48:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231627gitlab # [ 206.416264] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 991628gitlab # [ 206.492800] gitaly[892]: time="2026-10-10T10:48:03.648Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTM8FCN2V6JN1R6MJZSBC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.639" grpc.time_ms=8.58 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1629gitlab # [ 206.497329] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 531630gitlab # [ 206.540949] gitaly[892]: time="2026-10-10T10:48:03.696Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMA7ZVRFW6DHFGWFZGRZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.689" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1631gitlab # [ 206.541673] gitaly[892]: time="2026-10-10T10:48:03.696Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMA7ZVRFW6DHFGWFZGRZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.689" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1632gitlab # [ 206.542563] gitaly[892]: time="2026-10-10T10:48:03.697Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMA7ZVRFW6DHFGWFZGRZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.689" grpc.time_ms=8.12 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1633gitlab # [ 206.573977] gitlab-workhorse[828]: time="2026-10-10T10:48:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671634gitlab # [ 206.575743] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 731635gitlab # [ 206.659322] gitaly[892]: time="2026-10-10T10:48:03.815Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMDXHN7STQ1JXFG7NBXW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.808" grpc.time_ms=6.273 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1636gitlab # [ 206.685918] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 671637gitlab # [ 206.744975] gitaly[892]: time="2026-10-10T10:48:03.900Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMG5EV213BFK73M4JT4R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.882" pid=892 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1638gitlab # [ 206.745828] gitaly[892]: time="2026-10-10T10:48:03.900Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMG5EV213BFK73M4JT4R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.882" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1639gitlab # [ 206.746854] gitaly[892]: time="2026-10-10T10:48:03.901Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMG5EV213BFK73M4JT4R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:03.882" grpc.time_ms=18.181 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1640gitlab # [ 206.805071] gitlab-workhorse[828]: time="2026-10-10T10:48:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091641gitlab # [ 206.806951] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1141642gitlab # [ 206.882710] gitaly[892]: time="2026-10-10T10:48:04.037Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:34.036" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:04.036" grpc.time_ms=0.85 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1643gitlab # [ 206.934356] gitaly[892]: time="2026-10-10T10:48:04.090Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-10T10:48:34.080" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:04.081" grpc.time_ms=8.82 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1644gitlab # [ 206.981582] gitaly[892]: time="2026-10-10T10:48:04.137Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:14.119" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:04.129" grpc.time_ms=7.689 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1645gitlab # [ 207.006958] gitaly[892]: time="2026-10-10T10:48:04.162Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:14.144" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:04.155" grpc.time_ms=7.52 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1646gitlab # [ 207.059158] gitaly[892]: time="2026-10-10T10:48:04.214Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:14.204" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:04.214" grpc.time_ms=0.207 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1647gitlab # [ 207.074018] gitaly[892]: time="2026-10-10T10:48:04.229Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:48:14.212" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:04.222" grpc.time_ms=6.952 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1648gitlab # [ 207.117241] gitaly[892]: time="2026-10-10T10:48:04.273Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:04.271" grpc.time_ms=1.652 pid=892 span.kind=server system=grpc1649gitlab # [ 207.151073] gitaly[892]: time="2026-10-10T10:48:04.306Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:04.306" grpc.time_ms=0.326 pid=892 span.kind=server system=grpc1650gitlab # [ 207.168259] gitaly[892]: time="2026-10-10T10:48:04.324Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R diskcache=5e0d4a0b-4bde-4390-9733-d3e263074ac2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:14.233" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:04.243" pid=892 ref=refs/keep-around/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 remote_ip="2001:db8:1::1" revision=f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 span.kind=server system=grpc username=root1651gitlab # [ 207.178767] gitaly[892]: time="2026-10-10T10:48:04.324Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5300 command.oublock=8 command.real_time_ms=71 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:14.233" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:04.243" grpc.time_ms=80.205 pid=892 ref=refs/keep-around/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 remote_ip="2001:db8:1::1" revision=f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 span.kind=server system=grpc username=root1652gitlab # [ 207.179433] gitaly[892]: time="2026-10-10T10:48:04.329Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:34.328" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:04.328" grpc.time_ms=0.526 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653gitlab # [ 207.295388] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4551654gitlab # [ 207.306713] nixbot[2666]: {"ts": "2026-10-10T10:48:04.461Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-f294d2c18dd3-0"}1655gitlab # [ 207.490208] nixbot[2666]: {"ts": "2026-10-10T10:48:04.645Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-f294d2c18dd3-0", "argv0": "bwrap"}1656gitlab # [ 207.591669] systemd[1]: Started Nix Daemon.1657gitlab # [ 207.656790] nix-daemon[2824]: accepted connection from pid 2819, user nixbot1658gitlab # [ 211.794559] nix-daemon[2824]: accepted connection from pid 2867, user nixbot1659gitlab # [ 211.809603] nix-daemon[2824]: accepted connection from pid 2819, user nixbot1660gitlab # [ 216.661965] nix-daemon[2824]: accepted connection from pid 2819, user nixbot1661gitlab # [ 216.761103] nix-daemon[2824]: accepted connection from pid 2916, user nixbot1662gitlab # [ 216.805777] gitaly[892]: time="2026-10-10T10:48:13.960Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTYAGZY2ZQJ0MEYPRSHDM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:43.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:13.959" grpc.time_ms=0.894 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1663gitlab # [ 216.848559] gitaly[892]: time="2026-10-10T10:48:14.004Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTYAGZY2ZQJ0MEYPRSHDM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:23.147" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:13.997" grpc.time_ms=6.667 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1664gitlab # [ 216.887908] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1381665gitlab # [ 216.945107] gitaly[892]: time="2026-10-10T10:48:14.100Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTYF3A24B8E00T65PVKTS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:44.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:14.099" grpc.time_ms=0.978 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1666gitlab # [ 217.018321] gitaly[892]: time="2026-10-10T10:48:14.173Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTYF3A24B8E00T65PVKTS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:24.150" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:14.160" grpc.time_ms=12.96 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1667gitlab # [ 217.080297] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1841668gitlab # [ 221.206590] nix-daemon[2824]: accepted connection from pid 2971, user nixbot1669gitlab # [ 221.307747] gitaly[892]: time="2026-10-10T10:48:18.461Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV2PS93XQD7285Y4Y1EHC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:48.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:18.460" grpc.time_ms=0.874 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670gitlab # [ 221.506328] gitaly[892]: time="2026-10-10T10:48:18.661Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPV2PS93XQD7285Y4Y1EHC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:28.261" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:18.651" grpc.time_ms=10.158 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671gitlab # [ 221.586176] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3471672gitlab # [ 221.787318] nixbot[2666]: {"ts": "2026-10-10T10:48:18.942Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "f294d2c18dd3daa720f30e8a9f2e5d56f281fb60", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1673gitlab # [ 221.893880] gitaly[892]: time="2026-10-10T10:48:19.049Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPV39N26H10KEYMPSF9Q66 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.043" grpc.time_ms=6.146 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1674gitlab # [ 221.922604] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 521675gitlab # [ 221.942742] gitaly[892]: time="2026-10-10T10:48:19.098Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV3BBMXKR9VA7BYNXM82Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.089" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1676gitlab # [ 221.963277] gitaly[892]: time="2026-10-10T10:48:19.098Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV3BBMXKR9VA7BYNXM82Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.089" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1677gitlab # [ 221.963869] gitlab-workhorse[828]: time="2026-10-10T10:48:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671678gitlab # [ 221.964772] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 471679gitlab # [ 221.965704] gitaly[892]: time="2026-10-10T10:48:19.098Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV3BBMXKR9VA7BYNXM82Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.089" grpc.time_ms=8.908 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1680gitlab # [ 222.056674] gitaly[892]: time="2026-10-10T10:48:19.212Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPV3F4ZZ1RHFT3J7E97H4S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.204" grpc.time_ms=7.169 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1681gitlab # [ 222.068916] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 401682gitlab # [ 222.230292] gitaly[892]: time="2026-10-10T10:48:19.385Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV3GVJNT4QN03D2WJGSRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.373" pid=892 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1683gitlab # [ 222.231246] gitaly[892]: time="2026-10-10T10:48:19.386Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV3GVJNT4QN03D2WJGSRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.373" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1684gitlab # [ 222.231804] gitaly[892]: time="2026-10-10T10:48:19.386Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV3GVJNT4QN03D2WJGSRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:19.373" grpc.time_ms=13.326 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1685gitlab # [ 222.232977] gitlab-workhorse[828]: time="2026-10-10T10:48:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671686gitlab # [ 222.234442] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1601687gitlab # [ 223.887453] sidekiq-cluster[2540]: 2026-10-10T10:48:21.042Z pid=2540 tid=xtw INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2540, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>438616064}1688gitlab # [ 224.075917] sidekiq-cluster[2540]: W, [2026-10-10T10:48:21.231459 #2540] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1689gitlab # [ 224.161806] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:21.317Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1690gitlab # [ 225.900313] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:23.055Z","message":"GitLab reliable fetch activated!"}1691gitlab # [ 225.901422] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:23.055Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1692gitlab # [ 245.296837] postgres[3227]: [3227] ERROR: relation "table_without_models" does not exist1693gitlab # [ 245.299881] postgres[3227]: [3227] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1694gitlab # [ 247.737350] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:44.892Z","message":"Booted Rails 7.2.3.1 application in production environment"}1695gitlab # [ 247.739909] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:44.892Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1696gitlab # [ 247.741468] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:44.892Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1697gitlab # [ 247.742954] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:44.892Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1698gitlab # [ 247.744586] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:44.893Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1699gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 42.62 seconds)1700gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1701gitlab # [ 248.591455] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:45 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 391702gitlab # [ 248.692141] gitaly[892]: time="2026-10-10T10:48:45.847Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXDMH78PCYYD6YQ4V69Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:45.836" grpc.time_ms=10.819 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1703gitlab # [ 248.721105] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:45 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 991704gitlab # Enumerating objects: 5, done.1705gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1706gitlab # Delta compression using up to 4 threads1707gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1708gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done.1709gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1710gitlab # [ 248.985439] gitaly[892]: time="2026-10-10T10:48:46.141Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-10T10:49:16.028" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-10T10:48:46.128" grpc.time_ms=12.025 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711gitlab # [ 249.043575] gitaly[892]: time="2026-10-10T10:48:46.199Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S duration_ms=129ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/allowed"1712gitlab # [ 249.081366] gitaly[892]: time="2026-10-10T10:48:46.237Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S duration_ms=30ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/pre_receive"1713gitlab # [ 249.091365] gitaly[892]: time="2026-10-10T10:48:46.237Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:46.068" grpc.time_ms=168.551 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1714gitlab # [ 249.141841] gitaly[892]: time="2026-10-10T10:48:46.297Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:46.295" grpc.time_ms=1.617 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1715gitlab # [ 249.146506] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.301Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7a32ce792497256f1dcd2324","created_at":"2026-10-10T10:47:52.148Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.175Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7a32ce792497256f1dcd2324: start","job_status":"start","queue_duration_s":54.125399,"scheduling_latency_s":54.12545}1716gitlab # [ 249.215847] gitaly[892]: time="2026-10-10T10:48:46.371Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:46.369" grpc.time_ms=2.159 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1717gitlab # [ 249.252505] gitaly[892]: time="2026-10-10T10:48:46.408Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:46.407" grpc.time_ms=0.569 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1718gitlab # [ 249.296067] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.451Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7a32ce792497256f1dcd2324","created_at":"2026-10-10T10:47:52.148Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.175Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7a32ce792497256f1dcd2324: done: 0.149962 sec","job_status":"done","queue_duration_s":54.125399,"scheduling_latency_s":54.12545,"redis_calls":25,"redis_duration_s":0.004038,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002261,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082413547,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.149962,"completed_at":"2026-10-10T10:48:46.450Z","load_balancing_strategy":"primary","db_duration_s":0.059683,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1719gitlab # [ 249.297119] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.452Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"813b284c150f071f9a3d31bf","created_at":"2026-10-10T10:47:52.183Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.187Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-813b284c150f071f9a3d31bf: start","job_status":"start","queue_duration_s":54.264734,"scheduling_latency_s":54.264752}1720gitlab # [ 249.337064] gitaly[892]: time="2026-10-10T10:48:46.492Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S duration_ms=55ns method=POST pid=892 status=200 url="http://unix/api/v4/internal/post_receive"1721gitlab # [ 249.337744] gitaly[892]: time="2026-10-10T10:48:46.492Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:46.436" grpc.time_ms=56.383 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1722gitlab # [ 249.370310] gitaly[892]: time="2026-10-10T10:48:46.526Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S diskcache=ac5fbd9b-1708-46dc-878b-fda542f184f2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:45.982" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1723gitlab # [ 249.370920] gitaly[892]: time="2026-10-10T10:48:46.526Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=255 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13615 command.oublock=32 command.real_time_ms=532 command.spawn_token_fork_ms=0 command.system_time_ms=158 command.user_time_ms=97 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:45.982" grpc.time_ms=543.77 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1724gitlab # [ 249.371514] gitlab-workhorse[828]: time="2026-10-10T10:48:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521725gitlab # To http://gitlab/root/test-flake.git1726gitlab # [ 249.372160] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:46 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6281727gitlab # f294d2c..49594e0 master -> master[ 249.373219] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.526Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"813b284c150f071f9a3d31bf","created_at":"2026-10-10T10:47:52.183Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.187Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-813b284c150f071f9a3d31bf: done: 0.073892 sec","job_status":"done","queue_duration_s":54.264734,"scheduling_latency_s":54.264752,"redis_calls":14,"redis_duration_s":0.0020570000000000002,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000373,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.057973569,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.073892,"completed_at":"2026-10-10T10:48:46.525Z","load_balancing_strategy":"primary","db_duration_s":0.012854,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1728gitlab # 1729gitlab # [ 249.416230] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.531Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"bca6dcc6a3211a93d502799e","created_at":"2026-10-10T10:47:52.736Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.745Z","job_size_bytes":5,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-bca6dcc6a3211a93d502799e: start","job_status":"start","queue_duration_s":53.786028,"scheduling_latency_s":53.786085}1730gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.97 seconds)1731gitlab: must succeed: git -C /tmp/test-flake rev-parse master1732gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1733gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1734gitlab # [ 249.505322] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.660Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"bca6dcc6a3211a93d502799e","created_at":"2026-10-10T10:47:52.736Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.745Z","job_size_bytes":5,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-bca6dcc6a3211a93d502799e: done: 0.129747 sec","job_status":"done","queue_duration_s":53.786028,"scheduling_latency_s":53.786085,"redis_calls":12,"redis_duration_s":0.002258,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.104165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360102,"duration_s":0.129747,"completed_at":"2026-10-10T10:48:46.660Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015487,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1735gitlab # [ 249.524794] nixbot[2666]: {"ts": "2026-10-10T10:48:46.679Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57694 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1736gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.08 seconds)1737gitlab # [ 249.550142] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.705Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"20438a7bc6b2bab4470e5033","created_at":"2026-10-10T10:47:52.781Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.787Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-20438a7bc6b2bab4470e5033: start","job_status":"start","queue_duration_s":53.918381,"scheduling_latency_s":53.918419}1738gitlab # [ 249.714128] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.869Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"20438a7bc6b2bab4470e5033","created_at":"2026-10-10T10:47:52.781Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.787Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-20438a7bc6b2bab4470e5033: done: 0.164059 sec","job_status":"done","queue_duration_s":53.918381,"scheduling_latency_s":53.918419,"redis_calls":11,"redis_duration_s":0.002835,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000658,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001743,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.134,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.134,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000379657,"duration_s":0.164059,"completed_at":"2026-10-10T10:48:46.869Z","load_balancing_strategy":"primary","db_duration_s":0.075656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1739gitlab # [ 249.716831] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.871Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"87909a04029e48fc0704e9be","created_at":"2026-10-10T10:47:52.790Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.795Z","job_size_bytes":89,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-87909a04029e48fc0704e9be: start","job_status":"start","queue_duration_s":54.075089,"scheduling_latency_s":54.075104}1740gitlab # [ 249.799959] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.908Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"87909a04029e48fc0704e9be","created_at":"2026-10-10T10:47:52.790Z","trace_propagation_headers":{"sentry-trace":"4da84cc5740643a0a1c5a9adf15c4a68-f8e04ab2367d4019","baggage":"sentry-trace_id=4da84cc5740643a0a1c5a9adf15c4a68,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT88KNPECK2PDGZ1J2AXQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:52.795Z","job_size_bytes":89,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-87909a04029e48fc0704e9be: done: 0.037352 sec","job_status":"done","queue_duration_s":54.075089,"scheduling_latency_s":54.075104,"redis_calls":11,"redis_duration_s":0.003046,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000764,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000454807,"duration_s":0.037352,"completed_at":"2026-10-10T10:48:46.908Z","load_balancing_strategy":"primary","db_duration_s":0.007227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1741gitlab # [ 249.803826] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.911Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b86f43e8ca5de1a3ff92aa6b","created_at":"2026-10-10T10:47:59.814Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-6013b9479303442a","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:59.836Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b86f43e8ca5de1a3ff92aa6b: start","job_status":"start","queue_duration_s":47.074241,"scheduling_latency_s":47.074347}1742gitlab # [ 249.844303] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:46.937Z","message":"changes: \"0000000000000000000000000000000000000000 f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 refs/heads/master\\n\""}1743gitlab # [ 249.845670] gitaly[892]: time="2026-10-10T10:48:46.978Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:56.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:46.969" grpc.time_ms=8.851 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 249.846685] gitaly[892]: time="2026-10-10T10:48:46.987Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:56.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:46.981" grpc.time_ms=5.233 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 249.872763] gitaly[892]: time="2026-10-10T10:48:47.028Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-10T10:48:57.010" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.020" grpc.time_ms=7.689 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 249.919740] gitaly[892]: time="2026-10-10T10:48:47.075Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:47.075" grpc.time_ms=0.393 pid=892 span.kind=server system=grpc1747gitlab # [ 249.954764] gitaly[892]: time="2026-10-10T10:48:47.110Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:47.109" grpc.time_ms=1.102 pid=892 span.kind=server system=grpc1748gitlab # [ 249.974388] gitaly[892]: time="2026-10-10T10:48:47.130Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH diskcache=7a114172-d428-4245-9e9a-0cef54551ac7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:57.028" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:47.039" pid=892 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1749gitlab # [ 249.983792] gitaly[892]: time="2026-10-10T10:48:47.139Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4920 command.oublock=8 command.real_time_ms=81 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:57.028" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:47.039" grpc.time_ms=91.276 pid=892 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1750gitlab # [ 249.995905] gitaly[892]: time="2026-10-10T10:48:47.151Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:57.140" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:47.151" grpc.time_ms=0.151 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751gitlab # [ 250.007520] gitaly[892]: time="2026-10-10T10:48:47.163Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=937 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-10T10:48:57.143" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.154" grpc.time_ms=8.924 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752gitlab # [ 250.082733] gitaly[892]: time="2026-10-10T10:48:47.238Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=337 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-10T10:49:17.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.229" grpc.time_ms=8.231 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 250.124838] gitaly[892]: time="2026-10-10T10:48:47.280Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:57.269" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:47.279" grpc.time_ms=0.547 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754gitlab # [ 250.132663] gitaly[892]: time="2026-10-10T10:48:47.288Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-10T10:48:57.272" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-10T10:48:47.283" grpc.time_ms=5.086 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755gitlab # [ 250.155896] gitaly[892]: time="2026-10-10T10:48:47.311Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:57.292" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.302" grpc.time_ms=8.662 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 250.179152] gitaly[892]: time="2026-10-10T10:48:47.334Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:57.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.327" grpc.time_ms=7.693 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757gitlab # [ 250.203280] gitaly[892]: time="2026-10-10T10:48:47.359Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T10:49:17.148" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.349" grpc.time_ms=9.77 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1759gitlab # [ 250.586508] nixbot[2666]: {"ts": "2026-10-10T10:48:47.741Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57698 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1760gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.04 seconds)1761gitlab # [ 250.594850] gitaly[892]: time="2026-10-10T10:48:47.750Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-10T10:48:57.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.748" grpc.time_ms=1.773 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 250.623468] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:47.778Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b86f43e8ca5de1a3ff92aa6b","created_at":"2026-10-10T10:47:59.814Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-6013b9479303442a","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:59.836Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b86f43e8ca5de1a3ff92aa6b: done: 0.867594 sec","job_status":"done","queue_duration_s":47.074241,"scheduling_latency_s":47.074347,"gitaly_calls":13,"gitaly_duration_s":0.222151,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.040018,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003744,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005787,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001345,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004591,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.012114,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012437,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.188,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.21,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.529535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000692546,"exclusive_lock_hold_duration_s":0.027737324,"duration_s":0.867594,"completed_at":"2026-10-10T10:48:47.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.102461,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1763gitlab # [ 250.650778] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:47.779Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1824784c83f330d9dacaf488","created_at":"2026-10-10T10:48:04.332Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.339Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1824784c83f330d9dacaf488: start","job_status":"start","queue_duration_s":43.440138,"scheduling_latency_s":43.440154}1764gitlab # [ 250.725689] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:47.881Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1824784c83f330d9dacaf488","created_at":"2026-10-10T10:48:04.332Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.339Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1824784c83f330d9dacaf488: done: 0.101228 sec","job_status":"done","queue_duration_s":43.440138,"scheduling_latency_s":43.440154,"redis_calls":16,"redis_duration_s":0.005011,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002684,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034781,"duration_s":0.101228,"completed_at":"2026-10-10T10:48:47.881Z","load_balancing_strategy":"primary","db_duration_s":0.024014,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1765gitlab # [ 250.747370] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:47.882Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0a56355861e7d899781513e5","created_at":"2026-10-10T10:48:04.340Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.343Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0a56355861e7d899781513e5: start","job_status":"start","queue_duration_s":43.539546,"scheduling_latency_s":43.539566}1766gitlab # [ 250.829436] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:47.984Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0a56355861e7d899781513e5","created_at":"2026-10-10T10:48:04.340Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.343Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0a56355861e7d899781513e5: done: 0.101601 sec","job_status":"done","queue_duration_s":43.539546,"scheduling_latency_s":43.539566,"redis_calls":21,"redis_duration_s":0.005997000000000001,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002467,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001935,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000540012,"duration_s":0.101601,"completed_at":"2026-10-10T10:48:47.984Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1767gitlab # [ 250.872408] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:48.027Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"223f5bfde41520b369649d80","created_at":"2026-10-10T10:48:04.434Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.438Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-223f5bfde41520b369649d80: start","job_status":"start","queue_duration_s":43.588741,"scheduling_latency_s":43.588789}1768gitlab # [ 251.158514] gitaly[892]: time="2026-10-10T10:48:48.313Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.312" grpc.time_ms=0.803 pid=892 span.kind=server system=grpc1769gitlab # [ 251.215839] gitaly[892]: time="2026-10-10T10:48:48.371Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.370" grpc.time_ms=1.004 pid=892 span.kind=server system=grpc1770gitlab # [ 251.268190] gitaly[892]: time="2026-10-10T10:48:48.423Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.422" grpc.time_ms=0.736 pid=892 span.kind=server system=grpc1771gitlab # [ 251.286339] gitaly[892]: time="2026-10-10T10:48:48.442Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R diskcache=03158edd-d663-4112-a814-a36bc61c364d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.071" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.272" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772gitlab # [ 251.287737] gitaly[892]: time="2026-10-10T10:48:48.442Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=147 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7295 command.oublock=0 command.real_time_ms=160 command.spawn_token_fork_ms=0 command.system_time_ms=98 command.user_time_ms=49 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.071" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.272" grpc.time_ms=170.08 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 251.382208] gitaly[892]: time="2026-10-10T10:48:48.537Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:18.336" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.536" grpc.time_ms=1.218 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774gitlab # [ 251.438592] gitaly[892]: time="2026-10-10T10:48:48.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:18.392" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.593" grpc.time_ms=0.82 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 251.479346] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:48.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"223f5bfde41520b369649d80","created_at":"2026-10-10T10:48:04.434Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.438Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2540,"job_status":"rescheduled","message":"PipelineProcessWorker JID-223f5bfde41520b369649d80: rescheduled"}1776gitlab # [ 251.522439] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:48.677Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"223f5bfde41520b369649d80","created_at":"2026-10-10T10:48:04.434Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-3ea030bdecda462e","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:04.438Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-223f5bfde41520b369649d80: done: 0.650012 sec","job_status":"done","queue_duration_s":43.588741,"scheduling_latency_s":43.588789,"gitaly_calls":3,"gitaly_duration_s":0.231743,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.056665,"redis_read_bytes":460,"redis_write_bytes":29291,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001378,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001021,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010392,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004369,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15578,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.033982,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000691,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0006,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002854,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.022,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.282782,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.527372054,"duration_s":0.650012,"completed_at":"2026-10-10T10:48:48.677Z","load_balancing_strategy":"primary","db_duration_s":0.06942,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1777gitlab # [ 251.579322] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:48.734Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 49594e03ed656717508a1fe4aec0d95db48c51fc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"4281aa7ac91dafcef637450c","created_at":"2026-10-10T10:48:46.471Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-fcf4f92e2cb44643","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:46.473Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-4281aa7ac91dafcef637450c: start","job_status":"start","queue_duration_s":2.260988,"scheduling_latency_s":2.261035}1778gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1779gitlab # [ 251.611993] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:48.767Z","message":"changes: \"f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 49594e03ed656717508a1fe4aec0d95db48c51fc refs/heads/master\\n\""}1780gitlab # [ 251.633764] nixbot[2666]: {"ts": "2026-10-10T10:48:48.789Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57710 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1781gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.04 seconds)1782gitlab # [ 251.658832] gitaly[892]: time="2026-10-10T10:48:48.814Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:58.061" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.802" grpc.time_ms=12.036 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab # [ 251.676582] gitaly[892]: time="2026-10-10T10:48:48.832Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:58.080" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:48.820" grpc.time_ms=10.894 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1784gitlab # [ 251.684223] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:48.838Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"2964f48fdf52d5ee7281ed76","created_at":"2026-10-10T10:48:48.836Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"6ea6f6667d401438ad75337a","job_size_bytes":46,"pid":2540,"job_status":"deduplicated","message":"ProjectCacheWorker JID-2964f48fdf52d5ee7281ed76: deduplicated: until executing","deduplication.type":"until executing"}1785gitlab # [ 251.711408] gitaly[892]: time="2026-10-10T10:48:48.867Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-10T10:48:58.114" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.854" grpc.time_ms=12.094 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786gitlab # [ 251.731804] gitaly[892]: time="2026-10-10T10:48:48.887Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-10T10:49:18.675" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.875" grpc.time_ms=11.704 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 251.749310] gitaly[892]: time="2026-10-10T10:48:48.905Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-10T10:48:58.158" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-10T10:48:48.899" grpc.time_ms=5.616 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1788gitlab # [ 251.771446] gitaly[892]: time="2026-10-10T10:48:48.927Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:58.176" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.916" grpc.time_ms=10.708 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 251.855311] gitaly[892]: time="2026-10-10T10:48:49.010Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-10T10:48:59.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.009" grpc.time_ms=1.452 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 251.880855] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.028Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 49594e03ed656717508a1fe4aec0d95db48c51fc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"4281aa7ac91dafcef637450c","created_at":"2026-10-10T10:48:46.471Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-fcf4f92e2cb44643","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:46.473Z","job_size_bytes":185,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-4281aa7ac91dafcef637450c: done: 0.293807 sec","job_status":"done","queue_duration_s":2.260988,"scheduling_latency_s":2.261035,"gitaly_calls":7,"gitaly_duration_s":0.087366,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.025882,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.001972,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000892,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004031,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.011967,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.00702,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.154249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001511645,"exclusive_lock_hold_duration_s":0.03040582,"duration_s":0.293807,"completed_at":"2026-10-10T10:48:49.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028587,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1791gitlab # [ 251.889162] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.029Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6ea6f6667d401438ad75337a","created_at":"2026-10-10T10:48:46.988Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.004Z","job_size_bytes":46,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6ea6f6667d401438ad75337a: start","job_status":"start","queue_duration_s":2.025124,"scheduling_latency_s":2.025146}1792gitlab # [ 251.980346] gitaly[892]: time="2026-10-10T10:48:49.135Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-10T16:48:49.132" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:49.133" grpc.time_ms=2.332 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793gitlab # [ 252.007314] gitaly[892]: time="2026-10-10T10:48:49.163Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-10T10:49:19.051" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.152" grpc.time_ms=10.227 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1794gitlab # [ 252.048334] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.203Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6ea6f6667d401438ad75337a","created_at":"2026-10-10T10:48:46.988Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.004Z","job_size_bytes":46,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6ea6f6667d401438ad75337a: done: 0.173625 sec","job_status":"done","queue_duration_s":2.025124,"scheduling_latency_s":2.025146,"gitaly_calls":2,"gitaly_duration_s":0.027431,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008569,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002056,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000741,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001814,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003174,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000544762,"duration_s":0.173625,"completed_at":"2026-10-10T10:48:49.203Z","load_balancing_strategy":"primary","db_duration_s":0.015248,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1795gitlab # [ 252.052947] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.206Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"49eab83c9e3b5410b4b218d9","created_at":"2026-10-10T10:48:47.014Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.017Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-49eab83c9e3b5410b4b218d9: start","job_status":"start","queue_duration_s":2.189101,"scheduling_latency_s":2.189159}1796gitlab # [ 252.121372] gitaly[892]: time="2026-10-10T10:48:49.277Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH diskcache=b86b46f3-1005-461d-a513-390f8d8c973d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:49.240" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.241" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab # [ 252.122782] gitaly[892]: time="2026-10-10T10:48:49.277Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1285 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:49.240" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.241" grpc.time_ms=36.019 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798gitlab # [ 252.166373] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.321Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"49eab83c9e3b5410b4b218d9","created_at":"2026-10-10T10:48:47.014Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.017Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-49eab83c9e3b5410b4b218d9: done: 0.114951 sec","job_status":"done","queue_duration_s":2.189101,"scheduling_latency_s":2.189159,"gitaly_calls":1,"gitaly_duration_s":0.039601,"redis_calls":15,"redis_duration_s":0.004875,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000649,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000879,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00151,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08097006799999999,"duration_s":0.114951,"completed_at":"2026-10-10T10:48:49.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1799gitlab # [ 252.168892] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.323Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8f66ad85dd67c416bd7f74a1","created_at":"2026-10-10T10:48:47.147Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.149Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8f66ad85dd67c416bd7f74a1: start","job_status":"start","queue_duration_s":2.174681,"scheduling_latency_s":2.174707}1800gitlab # [ 252.203257] gitaly[892]: time="2026-10-10T10:48:49.358Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-10T10:49:19.245" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.346" grpc.time_ms=11.561 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 252.243630] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.398Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8f66ad85dd67c416bd7f74a1","created_at":"2026-10-10T10:48:47.147Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.149Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8f66ad85dd67c416bd7f74a1: done: 0.075244 sec","job_status":"done","queue_duration_s":2.174681,"scheduling_latency_s":2.174707,"gitaly_calls":1,"gitaly_duration_s":0.046567,"redis_calls":16,"redis_duration_s":0.00404,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000289,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001785,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075244,"completed_at":"2026-10-10T10:48:49.398Z","load_balancing_strategy":"primary","db_duration_s":0.003516,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1802gitlab # [ 252.245400] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:49.400Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"30c700164363600ed2fb809b","created_at":"2026-10-10T10:48:47.606Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6087bd1c96fb433e9a261b5e48ba1a096eb1684dc36c4dc9c8fd8202204a7975","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.612Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-30c700164363600ed2fb809b: start","job_status":"start","queue_duration_s":1.788655,"scheduling_latency_s":1.788675}1803gitlab # [ 252.290263] gitaly[892]: time="2026-10-10T10:48:49.422Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:19.320" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.420" grpc.time_ms=0.905 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 252.291049] gitaly[892]: time="2026-10-10T10:48:49.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:48:59.107" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:49.427" grpc.time_ms=9.43 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 252.362539] gitaly[892]: time="2026-10-10T10:48:49.518Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:19.415" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.516" grpc.time_ms=0.868 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 252.377922] gitaly[892]: time="2026-10-10T10:48:49.533Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=60 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-10T10:49:19.430" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.530" grpc.time_ms=2.697 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 252.496811] gitaly[892]: time="2026-10-10T10:48:49.652Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:59.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:49.640" grpc.time_ms=11.479 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1809gitlab # [ 252.681275] nixbot[2666]: {"ts": "2026-10-10T10:48:49.836Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57716 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1810gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.05 seconds)1811gitlab # [ 252.967599] gitaly[892]: time="2026-10-10T10:48:50.123Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T10:49:20.004" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:50.104" grpc.time_ms=18.206 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 252.990697] gitaly[892]: time="2026-10-10T10:48:50.146Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTG0HGYX54QCC73SYRZHH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-10T10:49:20.025" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:50.125" grpc.time_ms=20.993 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 253.051450] sidekiq-cluster[2540]: {"severity":"WARN","time":"2026-10-10T10:48:50.206Z","class":"CreatePipelineWorker","correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"f294d2c18dd3daa720f30e8a9f2e5d56f281fb60","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f294d2c18dd3daa720f30e8a9f2e5d56f281fb60","ref":"refs/heads/master"},"retry":0}1814gitlab # [ 253.056647] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.212Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"30c700164363600ed2fb809b","created_at":"2026-10-10T10:48:47.606Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6087bd1c96fb433e9a261b5e48ba1a096eb1684dc36c4dc9c8fd8202204a7975","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.612Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-30c700164363600ed2fb809b: done: 0.811423 sec","job_status":"done","queue_duration_s":1.788655,"scheduling_latency_s":1.788675,"gitaly_calls":7,"gitaly_duration_s":0.126645,"redis_calls":51,"redis_duration_s":0.011407,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001138,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004697,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003452,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.607521,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241372,"duration_s":0.811423,"completed_at":"2026-10-10T10:48:50.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06492,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1815gitlab # [ 253.112805] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.213Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"9c4297e3-a73f-47a4-a2f8-7b453e267775\"}"],"class":"WebHookWorker","jid":"c4ccd5443ccf64b2bcd2c4b6","created_at":"2026-10-10T10:48:47.706Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a8017f255de28c6a10ee39e9afcc3d1c151005fb82472184eb1f33442f1f472","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.711Z","job_size_bytes":1751,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c4ccd5443ccf64b2bcd2c4b6: start","job_status":"start","queue_duration_s":2.501819,"scheduling_latency_s":2.501838}1816gitlab # [ 253.246244] nixbot[2666]: {"ts": "2026-10-10T10:48:50.401Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "a9eeb300-33f8-4996-9bff-2cdbe6445914", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "f294d2c18dd3daa720f30e8a9f2e5d56f281fb60", "pr": null}1817gitlab # [ 253.252346] nixbot[2666]: {"ts": "2026-10-10T10:48:50.407Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:56988 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1818gitlab # [ 253.272327] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.427Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"9c4297e3-a73f-47a4-a2f8-7b453e267775\"}"],"class":"WebHookWorker","jid":"c4ccd5443ccf64b2bcd2c4b6","created_at":"2026-10-10T10:48:47.706Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-174b57733e8f4cae","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a8017f255de28c6a10ee39e9afcc3d1c151005fb82472184eb1f33442f1f472","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.711Z","job_size_bytes":1751,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c4ccd5443ccf64b2bcd2c4b6: done: 0.214283 sec","job_status":"done","queue_duration_s":2.501819,"scheduling_latency_s":2.501838,"redis_calls":19,"redis_duration_s":0.004341,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001082,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000471,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002039,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06634306899999842,"cpu_s":0.074576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000491683,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.214283,"completed_at":"2026-10-10T10:48:50.427Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1819gitlab # [ 253.348826] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.504Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fd61733c5a53da5ae089909c","created_at":"2026-10-10T10:48:48.258Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.262Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fd61733c5a53da5ae089909c: start","job_status":"start","queue_duration_s":2.24116,"scheduling_latency_s":2.241216}1820gitlab # [ 253.428950] gitaly[892]: time="2026-10-10T10:48:50.584Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW20CKA5AMAP79YBHGFPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.573" grpc.time_ms=10.071 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1821gitlab # [ 253.451235] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1401822gitlab # [ 253.461440] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.616Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fd61733c5a53da5ae089909c","created_at":"2026-10-10T10:48:48.258Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.262Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fd61733c5a53da5ae089909c: done: 0.113138 sec","job_status":"done","queue_duration_s":2.24116,"scheduling_latency_s":2.241216,"redis_calls":10,"redis_duration_s":0.005877,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00068,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000662,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000367085,"duration_s":0.113138,"completed_at":"2026-10-10T10:48:50.616Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019047,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1823gitlab # [ 253.483545] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.618Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"debfea94464ce5839bd76f0d","created_at":"2026-10-10T10:48:48.449Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.456Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-debfea94464ce5839bd76f0d: start","job_status":"start","queue_duration_s":2.162673,"scheduling_latency_s":2.162691}1824gitlab # [ 253.492453] gitaly[892]: time="2026-10-10T10:48:50.647Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW24TZEGE3R58CEP59CA1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.634" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1825gitlab # [ 253.493966] gitaly[892]: time="2026-10-10T10:48:50.647Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW24TZEGE3R58CEP59CA1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.634" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1826gitlab # [ 253.498907] gitlab-workhorse[828]: time="2026-10-10T10:48:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671827gitlab # [ 253.500582] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 611828gitlab # [ 253.502255] gitaly[892]: time="2026-10-10T10:48:50.648Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW24TZEGE3R58CEP59CA1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.634" grpc.time_ms=13.221 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1829gitlab # [ 253.566480] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.721Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"debfea94464ce5839bd76f0d","created_at":"2026-10-10T10:48:48.449Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.456Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-debfea94464ce5839bd76f0d: done: 0.103062 sec","job_status":"done","queue_duration_s":2.162673,"scheduling_latency_s":2.162691,"redis_calls":10,"redis_duration_s":0.002192,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.077,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000258133,"duration_s":0.103062,"completed_at":"2026-10-10T10:48:50.721Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1830gitlab # [ 253.608580] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.763Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d2b0b2245bd80b396a704e7c","created_at":"2026-10-10T10:48:48.472Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.478Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d2b0b2245bd80b396a704e7c: start","job_status":"start","queue_duration_s":2.285282,"scheduling_latency_s":2.285315}1831gitlab # [ 253.610566] gitaly[892]: time="2026-10-10T10:48:50.765Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW26TM524E7GGDSYKFZYH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.753" pid=892 remote_ip="2001:db8:1::1" request_sha=49fcfacc9616599e81ebe356f1598e4c82cda134 response_bytes=337 span.kind=server system=grpc1832gitlab # [ 253.611614] gitlab-workhorse[828]: time="2026-10-10T10:48:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371833gitlab # [ 253.612485] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 1151834gitlab # [ 253.613037] gitaly[892]: time="2026-10-10T10:48:50.765Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW26TM524E7GGDSYKFZYH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.753" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1835gitlab # [ 253.613623] gitaly[892]: time="2026-10-10T10:48:50.765Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW26TM524E7GGDSYKFZYH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:50.753" grpc.time_ms=12.237 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1836gitlab # [ 253.679264] nixbot[2666]: {"ts": "2026-10-10T10:48:50.834Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1837gitlab # [ 253.682266] nixbot[2666]: {"ts": "2026-10-10T10:48:50.836Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "f294d2c18dd3daa720f30e8a9f2e5d56f281fb60"}1838gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1839gitlab # [ 253.687878] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.843Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d2b0b2245bd80b396a704e7c","created_at":"2026-10-10T10:48:48.472Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.478Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d2b0b2245bd80b396a704e7c: done: 0.0796 sec","job_status":"done","queue_duration_s":2.285282,"scheduling_latency_s":2.285315,"redis_calls":10,"redis_duration_s":0.001725,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000293,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001039,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.048,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000285232,"duration_s":0.0796,"completed_at":"2026-10-10T10:48:50.843Z","load_balancing_strategy":"primary","db_duration_s":0.049018,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1840gitlab # [ 253.689086] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"67b1950237f18af183dce25f","created_at":"2026-10-10T10:48:48.488Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.491Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-67b1950237f18af183dce25f: start","job_status":"start","queue_duration_s":2.353043,"scheduling_latency_s":2.35306}1841gitlab # [ 253.741807] gitaly[892]: time="2026-10-10T10:48:50.897Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW2D4V6S1MPNA7013C29S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:20.396" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:50.896" grpc.time_ms=0.574 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 253.743260] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.898Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"67b1950237f18af183dce25f","created_at":"2026-10-10T10:48:48.488Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.491Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-67b1950237f18af183dce25f: done: 0.053908 sec","job_status":"done","queue_duration_s":2.353043,"scheduling_latency_s":2.35306,"redis_calls":13,"redis_duration_s":0.002442,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000387,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000229638,"duration_s":0.053908,"completed_at":"2026-10-10T10:48:50.898Z","load_balancing_strategy":"primary","db_duration_s":0.004608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1843gitlab # [ 253.791581] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.901Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a01e7f79523d68107ae5282f","created_at":"2026-10-10T10:48:48.492Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.494Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a01e7f79523d68107ae5282f: start","job_status":"start","queue_duration_s":2.407175,"scheduling_latency_s":2.407197}1844gitlab # [ 253.792576] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.943Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a01e7f79523d68107ae5282f","created_at":"2026-10-10T10:48:48.492Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.494Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a01e7f79523d68107ae5282f: done: 0.041337 sec","job_status":"done","queue_duration_s":2.407175,"scheduling_latency_s":2.407197,"redis_calls":10,"redis_duration_s":0.006076,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000319873,"duration_s":0.041337,"completed_at":"2026-10-10T10:48:50.943Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1845gitlab # [ 253.808637] nixbot[2666]: {"ts": "2026-10-10T10:48:50.918Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57002 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1846gitlab # [ 253.847458] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:50.946Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6f4c6c2e6d574fb0e6975344","created_at":"2026-10-10T10:48:48.495Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.498Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6f4c6c2e6d574fb0e6975344: start","job_status":"start","queue_duration_s":2.448108,"scheduling_latency_s":2.448125}1847gitlab # [ 253.848246] gitaly[892]: time="2026-10-10T10:48:50.998Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW2D4V6S1MPNA7013C29S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:00.128" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:50.989" grpc.time_ms=8.736 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1848gitlab # [ 253.854214] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.008Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6f4c6c2e6d574fb0e6975344","created_at":"2026-10-10T10:48:48.495Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.498Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6f4c6c2e6d574fb0e6975344: done: 0.06202 sec","job_status":"done","queue_duration_s":2.448108,"scheduling_latency_s":2.448125,"redis_calls":11,"redis_duration_s":0.017811,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.014093,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002314,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023355,"duration_s":0.06202,"completed_at":"2026-10-10T10:48:51.008Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1849gitlab # [ 253.877458] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.010Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"45cc310d0d2ab12b86bb107d","created_at":"2026-10-10T10:48:48.500Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.503Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-45cc310d0d2ab12b86bb107d: start","job_status":"start","queue_duration_s":2.506525,"scheduling_latency_s":2.506541}1850gitlab # [ 253.927285] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1961851gitlab # [ 253.935697] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.091Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"45cc310d0d2ab12b86bb107d","created_at":"2026-10-10T10:48:48.500Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.503Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-45cc310d0d2ab12b86bb107d: done: 0.081125 sec","job_status":"done","queue_duration_s":2.506525,"scheduling_latency_s":2.506541,"redis_calls":10,"redis_duration_s":0.00379,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000306185,"duration_s":0.081125,"completed_at":"2026-10-10T10:48:51.091Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1852gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.30 seconds)1853gitlab # [ 253.979120] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.134Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"23d188be25e63ef8309dda2e","created_at":"2026-10-10T10:48:48.504Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.507Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-23d188be25e63ef8309dda2e: start","job_status":"start","queue_duration_s":2.626687,"scheduling_latency_s":2.626726}1854gitlab # [ 254.018923] gitaly[892]: time="2026-10-10T10:48:51.174Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW2KDQ3KF8J9RHC4J8SNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:21.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.173" grpc.time_ms=0.625 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1855gitlab # [ 254.020322] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.175Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"23d188be25e63ef8309dda2e","created_at":"2026-10-10T10:48:48.504Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.507Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-23d188be25e63ef8309dda2e: done: 0.041783 sec","job_status":"done","queue_duration_s":2.626687,"scheduling_latency_s":2.626726,"redis_calls":10,"redis_duration_s":0.0021190000000000002,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001346,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00086715,"duration_s":0.041783,"completed_at":"2026-10-10T10:48:51.175Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1856gitlab # [ 254.030722] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.181Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4fbd71eda6fa0d1c5ef5aae7","created_at":"2026-10-10T10:48:48.511Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.514Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4fbd71eda6fa0d1c5ef5aae7: start","job_status":"start","queue_duration_s":2.667114,"scheduling_latency_s":2.667134}1857gitlab # [ 254.051118] gitaly[892]: time="2026-10-10T10:48:51.206Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW2KDQ3KF8J9RHC4J8SNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:01.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:51.200" grpc.time_ms=6.186 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 254.060666] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.215Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4fbd71eda6fa0d1c5ef5aae7","created_at":"2026-10-10T10:48:48.511Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.514Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4fbd71eda6fa0d1c5ef5aae7: done: 0.033787 sec","job_status":"done","queue_duration_s":2.667114,"scheduling_latency_s":2.667134,"redis_calls":10,"redis_duration_s":0.00255,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00051,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000363454,"duration_s":0.033787,"completed_at":"2026-10-10T10:48:51.215Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859gitlab # [ 254.099131] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"aa4c384f744fd4a16bcfd7d2","created_at":"2026-10-10T10:48:48.514Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.518Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-aa4c384f744fd4a16bcfd7d2: start","job_status":"start","queue_duration_s":2.69854,"scheduling_latency_s":2.698558}1860gitlab # [ 254.138697] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"aa4c384f744fd4a16bcfd7d2","created_at":"2026-10-10T10:48:48.514Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.518Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-aa4c384f744fd4a16bcfd7d2: done: 0.02095 sec","job_status":"done","queue_duration_s":2.69854,"scheduling_latency_s":2.698558,"redis_calls":10,"redis_duration_s":0.004360999999999999,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000745,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000325461,"duration_s":0.02095,"completed_at":"2026-10-10T10:48:51.237Z","load_balancing_strategy":"primary","db_duration_s":0.001734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1861gitlab # [ 254.152731] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f23044586fe43c2ea19a66d4","created_at":"2026-10-10T10:48:48.518Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.521Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f23044586fe43c2ea19a66d4: start","job_status":"start","queue_duration_s":2.718143,"scheduling_latency_s":2.718157}1862gitlab # [ 254.204993] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f23044586fe43c2ea19a66d4","created_at":"2026-10-10T10:48:48.518Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.521Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f23044586fe43c2ea19a66d4: done: 0.028318 sec","job_status":"done","queue_duration_s":2.718143,"scheduling_latency_s":2.718157,"redis_calls":10,"redis_duration_s":0.001571,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241371,"duration_s":0.028318,"completed_at":"2026-10-10T10:48:51.267Z","load_balancing_strategy":"primary","db_duration_s":0.00562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1863gitlab # [ 254.206322] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2101864gitlab # [ 254.207691] gitaly[892]: time="2026-10-10T10:48:51.314Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW2T52HR17JS2V75VKAW0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:21.313" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.313" grpc.time_ms=0.505 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 254.219489] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.268Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"bd66f730fe13e12f3c1fa382","created_at":"2026-10-10T10:48:48.524Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.527Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-bd66f730fe13e12f3c1fa382: start","job_status":"start","queue_duration_s":2.740802,"scheduling_latency_s":2.740817}1866gitlab # [ 254.259670] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.326Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"bd66f730fe13e12f3c1fa382","created_at":"2026-10-10T10:48:48.524Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.527Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-bd66f730fe13e12f3c1fa382: done: 0.057915 sec","job_status":"done","queue_duration_s":2.740802,"scheduling_latency_s":2.740817,"redis_calls":14,"redis_duration_s":0.005495,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00366,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001548,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000881118,"duration_s":0.057915,"completed_at":"2026-10-10T10:48:51.326Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867gitlab # [ 254.303029] gitaly[892]: time="2026-10-10T10:48:51.358Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW2T52HR17JS2V75VKAW0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:01.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:51.351" grpc.time_ms=6.998 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 254.303753] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f294d2c18dd3daa720f30e8a9f2e5d56f281fb60 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1321869gitlab # [ 254.304276] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.327Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"09b5654ff9d630a6b034e67e","created_at":"2026-10-10T10:48:48.635Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.665Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-09b5654ff9d630a6b034e67e: start","job_status":"start","queue_duration_s":2.662014,"scheduling_latency_s":2.662031}1870gitlab # [ 254.304844] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.427Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c7d2f0cfca8088adfc7b753","created_at":"2026-10-10T10:48:51.425Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-4da4ee1d5bcc49df","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"09b5654ff9d630a6b034e67e","job_size_bytes":3,"pid":2540,"job_status":"deduplicated","message":"PipelineProcessWorker JID-6c7d2f0cfca8088adfc7b753: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1871gitlab # [ 254.348148] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.428Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"09b5654ff9d630a6b034e67e","created_at":"2026-10-10T10:48:48.635Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.665Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2540,"job_status":"rescheduled","message":"PipelineProcessWorker JID-09b5654ff9d630a6b034e67e: rescheduled"}1872gitlab # [ 254.348773] gitaly[892]: time="2026-10-10T10:48:51.395Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:21.294" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.394" grpc.time_ms=0.791 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 254.349469] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.432Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"09b5654ff9d630a6b034e67e","created_at":"2026-10-10T10:48:48.635Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-fb077721bff745ac","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.665Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-09b5654ff9d630a6b034e67e: done: 0.104797 sec","job_status":"done","queue_duration_s":2.662014,"scheduling_latency_s":2.662031,"gitaly_calls":2,"gitaly_duration_s":0.005268,"redis_calls":37,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.005625,"redis_read_bytes":139,"redis_write_bytes":6670,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000657,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000465,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_calls":4,"redis_queues_duration_s":0.000382,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2220,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.002247,"redis_queues_metadata_read_bytes":89,"redis_queues_metadata_write_bytes":1432,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000189,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000175,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067769,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.075172124,"duration_s":0.104797,"completed_at":"2026-10-10T10:48:51.432Z","load_balancing_strategy":"primary","db_duration_s":0.030876,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1874gitlab # [ 254.422802] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.433Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1afe1c662e72e2a23379459e","created_at":"2026-10-10T10:48:48.846Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"49eab83c9e3b5410b4b218d9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.848Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1afe1c662e72e2a23379459e: start","job_status":"start","queue_duration_s":2.584794,"scheduling_latency_s":2.58481}1875gitlab # [ 254.423600] gitaly[892]: time="2026-10-10T10:48:51.421Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:21.320" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.420" grpc.time_ms=0.572 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 254.424265] gitaly[892]: time="2026-10-10T10:48:51.455Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S diskcache=b9b493cc-fb4b-464f-ae34-347f8bb1e210 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:51.444" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.445" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 254.424851] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.468Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1afe1c662e72e2a23379459e","created_at":"2026-10-10T10:48:48.846Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"49eab83c9e3b5410b4b218d9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.848Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1afe1c662e72e2a23379459e: done: 0.034735 sec","job_status":"done","queue_duration_s":2.584794,"scheduling_latency_s":2.58481,"gitaly_calls":1,"gitaly_duration_s":0.011945,"redis_calls":12,"redis_duration_s":0.00341,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000405,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.021651075,"duration_s":0.034735,"completed_at":"2026-10-10T10:48:51.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1878gitlab # [ 254.470400] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.469Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1ec4b985e14c04a1e6744550","created_at":"2026-10-10T10:48:48.976Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f156b43e72a4abf4ad22b678da351fb27463245030db4474e33bb65d75e9a63f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.979Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1ec4b985e14c04a1e6744550: start","job_status":"start","queue_duration_s":2.489525,"scheduling_latency_s":2.489539}1879gitlab # [ 254.515388] gitaly[892]: time="2026-10-10T10:48:51.455Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:51.444" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.445" grpc.time_ms=10.313 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 254.516050] gitaly[892]: time="2026-10-10T10:48:51.484Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:21.382" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.483" grpc.time_ms=0.534 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881gitlab # [ 254.516727] gitaly[892]: time="2026-10-10T10:48:51.490Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:49:01.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:51.485" grpc.time_ms=4.954 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882gitlab # [ 254.517385] gitaly[892]: time="2026-10-10T10:48:51.516Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:21.415" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.515" grpc.time_ms=0.63 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab # [ 254.523139] gitaly[892]: time="2026-10-10T10:48:51.521Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=63 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-10T10:49:21.418" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.518" grpc.time_ms=2.074 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 254.537897] gitaly[892]: time="2026-10-10T10:48:51.545Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:01.056" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:51.536" grpc.time_ms=9.11 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 254.646532] gitaly[892]: time="2026-10-10T10:48:51.802Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW2ZE62WPNVYJBP6MYTY3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:51.793" grpc.time_ms=8.164 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1886gitlab # [ 254.648345] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 3651887gitlab # [ 254.698264] gitaly[892]: time="2026-10-10T10:48:51.854Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3AY71AY8KACVQ60JJJJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:51.846" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1888gitlab # [ 254.707970] gitaly[892]: time="2026-10-10T10:48:51.854Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3AY71AY8KACVQ60JJJJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:51.846" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1889gitlab # [ 254.708617] gitlab-workhorse[828]: time="2026-10-10T10:48:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671890gitlab # [ 254.709098] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 501891gitlab # [ 254.709564] gitaly[892]: time="2026-10-10T10:48:51.854Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3AY71AY8KACVQ60JJJJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:51.846" grpc.time_ms=7.931 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1892gitlab # [ 254.753977] gitaly[892]: time="2026-10-10T10:48:51.909Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T10:49:21.800" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.900" grpc.time_ms=9.032 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 254.778867] gitaly[892]: time="2026-10-10T10:48:51.934Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVXJ9WD5E4K9S3VEX813S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-10T10:49:21.825" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.925" grpc.time_ms=8.594 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab # [ 254.790103] sidekiq-cluster[2540]: {"severity":"WARN","time":"2026-10-10T10:48:51.943Z","class":"CreatePipelineWorker","correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"f294d2c18dd3daa720f30e8a9f2e5d56f281fb60","after":"49594e03ed656717508a1fe4aec0d95db48c51fc","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"49594e03ed656717508a1fe4aec0d95db48c51fc","ref":"refs/heads/master"},"retry":0}1895gitlab # [ 254.795206] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.950Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1ec4b985e14c04a1e6744550","created_at":"2026-10-10T10:48:48.976Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f156b43e72a4abf4ad22b678da351fb27463245030db4474e33bb65d75e9a63f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.979Z","job_size_bytes":467,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1ec4b985e14c04a1e6744550: done: 0.481123 sec","job_status":"done","queue_duration_s":2.489525,"scheduling_latency_s":2.489539,"gitaly_calls":7,"gitaly_duration_s":0.060519,"redis_calls":33,"redis_duration_s":0.003916,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000396,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001732,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000768,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.402635,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.481123,"completed_at":"2026-10-10T10:48:51.950Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013205,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1896gitlab # [ 254.799622] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:51.952Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"5893af12-99a6-49a1-9076-3fc568ad2462\"}"],"class":"WebHookWorker","jid":"6669a4f1fa0672f7dbee68ec","created_at":"2026-10-10T10:48:48.996Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dc68a5ca75ed904af1f97b3cb8a3622408cabceb307d4248ee96f263b7c9f741","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.998Z","job_size_bytes":1749,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-6669a4f1fa0672f7dbee68ec: start","job_status":"start","queue_duration_s":2.95355,"scheduling_latency_s":2.953588}1897gitlab # [ 254.859455] nixbot[2666]: {"ts": "2026-10-10T10:48:52.014Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "1913233f-eef5-422b-bad5-656540bf31b0", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "49594e03ed656717508a1fe4aec0d95db48c51fc", "pr": null}1898gitlab # [ 254.867154] nixbot[2666]: {"ts": "2026-10-10T10:48:52.022Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57016 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1899gitlab # [ 254.876867] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.032Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"5893af12-99a6-49a1-9076-3fc568ad2462\"}"],"class":"WebHookWorker","jid":"6669a4f1fa0672f7dbee68ec","created_at":"2026-10-10T10:48:48.996Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-4d9e795726c54a58","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dc68a5ca75ed904af1f97b3cb8a3622408cabceb307d4248ee96f263b7c9f741","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.998Z","job_size_bytes":1749,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-6669a4f1fa0672f7dbee68ec: done: 0.080237 sec","job_status":"done","queue_duration_s":2.95355,"scheduling_latency_s":2.953588,"redis_calls":14,"redis_duration_s":0.001961,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000236,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000654,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.011439722000005759,"cpu_s":0.021128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.080237,"completed_at":"2026-10-10T10:48:52.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1900gitlab # [ 254.879627] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.033Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"7cfafc79e0079fa879a56037","created_at":"2026-10-10T10:48:49.178Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-8719c214fad64cd6","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:49.186Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-7cfafc79e0079fa879a56037: start","job_status":"start","queue_duration_s":2.847568,"scheduling_latency_s":2.847593}1901gitlab # [ 254.923066] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.070Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"7cfafc79e0079fa879a56037","created_at":"2026-10-10T10:48:49.178Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-8719c214fad64cd6","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:49.186Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-7cfafc79e0079fa879a56037: done: 0.036975 sec","job_status":"done","queue_duration_s":2.847568,"scheduling_latency_s":2.847593,"redis_calls":19,"redis_duration_s":0.003372,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00079,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.0004,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005131377,"duration_s":0.036975,"completed_at":"2026-10-10T10:48:52.070Z","load_balancing_strategy":"primary","db_duration_s":0.005752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1902gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1903gitlab # [ 254.934640] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.071Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c4ccd5443ccf64b2bcd2c4b6"],"class":"WebHooks::LogExecutionWorker","jid":"6052370f15725ccab7d388cf","created_at":"2026-10-10T10:48:50.411Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-80109d9850694790","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b8e14bc87918be2ef4d1bbae6ef9b22d812ca9d301efcdff3a4ba0b90e996d1b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:50.419Z","job_size_bytes":2593,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-6052370f15725ccab7d388cf: start","job_status":"start","queue_duration_s":1.651926,"scheduling_latency_s":1.651941}1904gitlab # [ 254.973516] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.111Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c4ccd5443ccf64b2bcd2c4b6"],"class":"WebHooks::LogExecutionWorker","jid":"6052370f15725ccab7d388cf","created_at":"2026-10-10T10:48:50.411Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-80109d9850694790","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b8e14bc87918be2ef4d1bbae6ef9b22d812ca9d301efcdff3a4ba0b90e996d1b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:50.419Z","job_size_bytes":2593,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-6052370f15725ccab7d388cf: done: 0.03946 sec","job_status":"done","queue_duration_s":1.651926,"scheduling_latency_s":1.651941,"redis_calls":10,"redis_duration_s":0.005048,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003813,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000256737,"duration_s":0.03946,"completed_at":"2026-10-10T10:48:52.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1905gitlab # [ 255.021767] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.112Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"bf90e727e73eb430b37ddac5","created_at":"2026-10-10T10:48:51.377Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-4da4ee1d5bcc49df","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:51.379Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bf90e727e73eb430b37ddac5: start","job_status":"start","queue_duration_s":0.732917,"scheduling_latency_s":0.732932}1906gitlab # [ 255.022682] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.147Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"bf90e727e73eb430b37ddac5","created_at":"2026-10-10T10:48:51.377Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-4da4ee1d5bcc49df","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:51.379Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-bf90e727e73eb430b37ddac5: done: 0.035432 sec","job_status":"done","queue_duration_s":0.732917,"scheduling_latency_s":0.732932,"redis_calls":7,"redis_duration_s":0.0012879999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035432,"completed_at":"2026-10-10T10:48:52.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1907gitlab # [ 255.077885] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.149Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e218712ea1623ce7afbab755","created_at":"2026-10-10T10:48:51.428Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-4da4ee1d5bcc49df","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:51.429Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e218712ea1623ce7afbab755: start","job_status":"start","queue_duration_s":0.720281,"scheduling_latency_s":0.720325}1908gitlab # [ 255.078714] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1121909gitlab # [ 255.079412] gitlab-workhorse[828]: time="2026-10-10T10:48:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671910gitlab # [ 255.080085] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 651911gitlab # [ 255.080506] gitaly[892]: time="2026-10-10T10:48:52.151Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW3J7CZQSEF5QZE1YF31K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.144" grpc.time_ms=6.52 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1912gitlab # [ 255.081224] gitaly[892]: time="2026-10-10T10:48:52.218Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3NSJN4NENW7R33T1J0F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.207" pid=892 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1913gitlab # [ 255.081821] nixbot[2666]: {"ts": "2026-10-10T10:48:52.213Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57022 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1914gitlab # [ 255.082873] gitaly[892]: time="2026-10-10T10:48:52.218Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3NSJN4NENW7R33T1J0F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.207" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1915gitlab # [ 255.130048] gitaly[892]: time="2026-10-10T10:48:52.218Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3NSJN4NENW7R33T1J0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.207" grpc.time_ms=11.188 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1916gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.17 seconds)1917gitlab # [ 255.153074] gitaly[892]: time="2026-10-10T10:48:52.308Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:52.308" grpc.time_ms=0.303 pid=892 span.kind=server system=grpc1918gitlab # [ 255.181643] gitaly[892]: time="2026-10-10T10:48:52.337Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:52.336" grpc.time_ms=0.456 pid=892 span.kind=server system=grpc1919gitlab # [ 255.211775] gitaly[892]: time="2026-10-10T10:48:52.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:52.367" grpc.time_ms=0.376 pid=892 span.kind=server system=grpc1920gitlab # [ 255.219664] gitaly[892]: time="2026-10-10T10:48:52.375Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW3V8BE7KVR48ZZE6529N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.369" grpc.time_ms=5.697 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1921gitlab # [ 255.224208] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 481922gitlab # [ 255.242521] gitaly[892]: time="2026-10-10T10:48:52.398Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R diskcache=a205173e-da71-45d5-8fed-5a92ef054485 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:22.184" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:52.285" pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 255.251567] gitaly[892]: time="2026-10-10T10:48:52.407Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=92 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7150 command.oublock=0 command.real_time_ms=105 command.spawn_token_fork_ms=0 command.system_time_ms=69 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:22.184" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:52.285" grpc.time_ms=112.864 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 255.267220] gitaly[892]: time="2026-10-10T10:48:52.422Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3WV0YYMGKKNN3SQB5PZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.410" pid=892 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1925gitlab # [ 255.276859] gitaly[892]: time="2026-10-10T10:48:52.423Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3WV0YYMGKKNN3SQB5PZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.410" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1926gitlab # [ 255.277728] gitlab-workhorse[828]: time="2026-10-10T10:48:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161927gitlab # [ 255.278472] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 441928gitlab # [ 255.279631] gitaly[892]: time="2026-10-10T10:48:52.423Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3WV0YYMGKKNN3SQB5PZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:52.410" grpc.time_ms=12.289 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc1929gitlab # [ 255.306745] gitaly[892]: time="2026-10-10T10:48:52.462Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:22.360" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:52.460" grpc.time_ms=1.877 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 255.343740] gitaly[892]: time="2026-10-10T10:48:52.499Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTMMTR7GF3X2GQTJFQR1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:22.398" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:52.498" grpc.time_ms=0.63 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931gitlab # [ 255.368188] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.523Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"e218712ea1623ce7afbab755","created_at":"2026-10-10T10:48:51.428Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-4da4ee1d5bcc49df","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:51.429Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e218712ea1623ce7afbab755: done: 0.373958 sec","job_status":"done","queue_duration_s":0.720281,"scheduling_latency_s":0.720325,"gitaly_calls":3,"gitaly_duration_s":0.165762,"redis_calls":71,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.052701,"redis_read_bytes":153,"redis_write_bytes":19879,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.012827,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000696,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":23,"redis_queues_duration_s":0.004439,"redis_queues_read_bytes":27,"redis_queues_write_bytes":13381,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.00346,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000232,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000856,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.017364,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095562,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.315456015,"duration_s":0.373958,"completed_at":"2026-10-10T10:48:52.523Z","load_balancing_strategy":"primary","db_duration_s":0.068546,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1932gitlab # [ 255.374311] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.525Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","6669a4f1fa0672f7dbee68ec"],"class":"WebHooks::LogExecutionWorker","jid":"b6ec145c171fc47b9a3f7407","created_at":"2026-10-10T10:48:52.024Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-bf857eef284f49f6","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:46b84092b181f435845dc1eb795644bf620b5fbe94e43f3ae743446c6c13044a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.027Z","job_size_bytes":2591,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b6ec145c171fc47b9a3f7407: start","job_status":"start","queue_duration_s":0.49783,"scheduling_latency_s":0.497858}1933gitlab # [ 255.418314] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.546Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","6669a4f1fa0672f7dbee68ec"],"class":"WebHooks::LogExecutionWorker","jid":"b6ec145c171fc47b9a3f7407","created_at":"2026-10-10T10:48:52.024Z","trace_propagation_headers":{"sentry-trace":"0ca8f3718c924b4ba614021513e97b87-bf857eef284f49f6","baggage":"sentry-trace_id=0ca8f3718c924b4ba614021513e97b87,sentry-release=Unknown"},"correlation_id":"01M4JPVXJ9WD5E4K9S3VEX813S","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:46b84092b181f435845dc1eb795644bf620b5fbe94e43f3ae743446c6c13044a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.027Z","job_size_bytes":2591,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b6ec145c171fc47b9a3f7407: done: 0.021242 sec","job_status":"done","queue_duration_s":0.49783,"scheduling_latency_s":0.497858,"redis_calls":7,"redis_duration_s":0.001248,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021242,"completed_at":"2026-10-10T10:48:52.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1934gitlab # [ 255.427862] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.547Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ce4abf3564b1769cf3b85e93","created_at":"2026-10-10T10:48:52.061Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-d776c22cd2044a38","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.064Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ce4abf3564b1769cf3b85e93: start","job_status":"start","queue_duration_s":0.482617,"scheduling_latency_s":0.482632}1935gitlab # [ 255.457565] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ce4abf3564b1769cf3b85e93","created_at":"2026-10-10T10:48:52.061Z","trace_propagation_headers":{"sentry-trace":"1292d3b30894455f9bf4e4c19121923d-d776c22cd2044a38","baggage":"sentry-trace_id=1292d3b30894455f9bf4e4c19121923d,sentry-release=Unknown"},"correlation_id":"01M4JPTG0HGYX54QCC73SYRZHH","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.064Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ce4abf3564b1769cf3b85e93: done: 0.065574 sec","job_status":"done","queue_duration_s":0.482617,"scheduling_latency_s":0.482632,"redis_calls":11,"redis_duration_s":0.00197,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000306,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001322,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267632,"duration_s":0.065574,"completed_at":"2026-10-10T10:48:52.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1936gitlab # [ 255.509294] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.614Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"78c6e2ff22915c3e39b5a9e2","created_at":"2026-10-10T10:48:52.277Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.280Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-78c6e2ff22915c3e39b5a9e2: start","job_status":"start","queue_duration_s":0.334008,"scheduling_latency_s":0.334024}1937gitlab # [ 255.510490] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.660Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"78c6e2ff22915c3e39b5a9e2","created_at":"2026-10-10T10:48:52.277Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.280Z","job_size_bytes":23,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-78c6e2ff22915c3e39b5a9e2: done: 0.046689 sec","job_status":"done","queue_duration_s":0.334008,"scheduling_latency_s":0.334024,"redis_calls":7,"redis_duration_s":0.001556,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046689,"completed_at":"2026-10-10T10:48:52.660Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007909,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1938gitlab # [ 255.550844] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.662Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9b2ec8bb145aa1b4829e1571","created_at":"2026-10-10T10:48:52.423Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.425Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9b2ec8bb145aa1b4829e1571: start","job_status":"start","queue_duration_s":0.236551,"scheduling_latency_s":0.236575}1939gitlab # [ 255.551668] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.681Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9b2ec8bb145aa1b4829e1571","created_at":"2026-10-10T10:48:52.423Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.425Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9b2ec8bb145aa1b4829e1571: done: 0.01928 sec","job_status":"done","queue_duration_s":0.236551,"scheduling_latency_s":0.236575,"redis_calls":7,"redis_duration_s":0.00124,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01928,"completed_at":"2026-10-10T10:48:52.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1940gitlab # [ 255.601706] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.682Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"926264300d0a29b44c6f1463","created_at":"2026-10-10T10:48:52.431Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.432Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-926264300d0a29b44c6f1463: start","job_status":"start","queue_duration_s":0.24969,"scheduling_latency_s":0.249705}1941gitlab # [ 255.602670] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.706Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"926264300d0a29b44c6f1463","created_at":"2026-10-10T10:48:52.431Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.432Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-926264300d0a29b44c6f1463: done: 0.024065 sec","job_status":"done","queue_duration_s":0.24969,"scheduling_latency_s":0.249705,"redis_calls":7,"redis_duration_s":0.001742,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024065,"completed_at":"2026-10-10T10:48:52.706Z","load_balancing_strategy":"primary","db_duration_s":0.004608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 255.642681] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.709Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"29b24dcf78ab4f44b20f5607","created_at":"2026-10-10T10:48:52.440Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.442Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-29b24dcf78ab4f44b20f5607: start","job_status":"start","queue_duration_s":0.267329,"scheduling_latency_s":0.267371}1943gitlab # [ 255.654149] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"29b24dcf78ab4f44b20f5607","created_at":"2026-10-10T10:48:52.440Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.442Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-29b24dcf78ab4f44b20f5607: done: 0.044883 sec","job_status":"done","queue_duration_s":0.267329,"scheduling_latency_s":0.267371,"redis_calls":10,"redis_duration_s":0.003203,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000951,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000611,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044883,"completed_at":"2026-10-10T10:48:52.754Z","load_balancing_strategy":"primary","db_duration_s":0.007079,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 255.701734] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.756Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6d1916d67155e62afbdf6e03","created_at":"2026-10-10T10:48:52.442Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.443Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6d1916d67155e62afbdf6e03: start","job_status":"start","queue_duration_s":0.312618,"scheduling_latency_s":0.312647}1945gitlab # [ 255.707638] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.774Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6d1916d67155e62afbdf6e03","created_at":"2026-10-10T10:48:52.442Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.443Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6d1916d67155e62afbdf6e03: done: 0.018864 sec","job_status":"done","queue_duration_s":0.312618,"scheduling_latency_s":0.312647,"redis_calls":7,"redis_duration_s":0.001673,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000513,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018864,"completed_at":"2026-10-10T10:48:52.774Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1946gitlab # [ 255.749166] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.776Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"19a990df5e243c66db176455","created_at":"2026-10-10T10:48:52.443Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.445Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-19a990df5e243c66db176455: start","job_status":"start","queue_duration_s":0.331049,"scheduling_latency_s":0.331063}1947gitlab # [ 255.760514] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.793Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"19a990df5e243c66db176455","created_at":"2026-10-10T10:48:52.443Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.445Z","job_size_bytes":19,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-19a990df5e243c66db176455: done: 0.017455 sec","job_status":"done","queue_duration_s":0.331049,"scheduling_latency_s":0.331063,"redis_calls":8,"redis_duration_s":0.001849,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0014,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017455,"completed_at":"2026-10-10T10:48:52.793Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1948gitlab # [ 255.813787] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.794Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ff093f1872dd0bbf7337c413","created_at":"2026-10-10T10:48:52.445Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.446Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ff093f1872dd0bbf7337c413: start","job_status":"start","queue_duration_s":0.348073,"scheduling_latency_s":0.348087}1949gitlab # [ 255.814411] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.811Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ff093f1872dd0bbf7337c413","created_at":"2026-10-10T10:48:52.445Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.446Z","job_size_bytes":66,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ff093f1872dd0bbf7337c413: done: 0.016367 sec","job_status":"done","queue_duration_s":0.348073,"scheduling_latency_s":0.348087,"redis_calls":7,"redis_duration_s":0.001709,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016367,"completed_at":"2026-10-10T10:48:52.811Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1950gitlab # [ 255.853851] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"51f612825bdbaa622af77101","created_at":"2026-10-10T10:48:52.447Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.448Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-51f612825bdbaa622af77101: start","job_status":"start","queue_duration_s":0.364165,"scheduling_latency_s":0.364179}1951gitlab # [ 255.895741] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.836Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"51f612825bdbaa622af77101","created_at":"2026-10-10T10:48:52.447Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.448Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-51f612825bdbaa622af77101: done: 0.024048 sec","job_status":"done","queue_duration_s":0.364165,"scheduling_latency_s":0.364179,"redis_calls":7,"redis_duration_s":0.00167,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024048,"completed_at":"2026-10-10T10:48:52.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1952gitlab # [ 255.937511] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.837Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"576f3f1dfcfb4bd27ac2f83e","created_at":"2026-10-10T10:48:52.450Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.451Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-576f3f1dfcfb4bd27ac2f83e: start","job_status":"start","queue_duration_s":0.386218,"scheduling_latency_s":0.386233}1953gitlab # [ 255.938211] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.856Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"576f3f1dfcfb4bd27ac2f83e","created_at":"2026-10-10T10:48:52.450Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.451Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-576f3f1dfcfb4bd27ac2f83e: done: 0.018539 sec","job_status":"done","queue_duration_s":0.386218,"scheduling_latency_s":0.386233,"redis_calls":7,"redis_duration_s":0.001605,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018539,"completed_at":"2026-10-10T10:48:52.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005383,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1954gitlab # [ 255.993301] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c214797e2423292fbab720bf","created_at":"2026-10-10T10:48:52.451Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.452Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c214797e2423292fbab720bf: start","job_status":"start","queue_duration_s":0.404807,"scheduling_latency_s":0.404822}1955gitlab # [ 255.993990] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.867Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c214797e2423292fbab720bf","created_at":"2026-10-10T10:48:52.451Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.452Z","job_size_bytes":6,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c214797e2423292fbab720bf: done: 0.009665 sec","job_status":"done","queue_duration_s":0.404807,"scheduling_latency_s":0.404822,"redis_calls":7,"redis_duration_s":0.001467,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001045,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009665,"completed_at":"2026-10-10T10:48:52.867Z","load_balancing_strategy":"primary","db_duration_s":0.000534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 256.045699] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.868Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2701ed23e7bba3716a49f037","created_at":"2026-10-10T10:48:52.452Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.453Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2701ed23e7bba3716a49f037: start","job_status":"start","queue_duration_s":0.414394,"scheduling_latency_s":0.414407}1957gitlab # [ 256.046507] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:52.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2701ed23e7bba3716a49f037","created_at":"2026-10-10T10:48:52.452Z","trace_propagation_headers":{"sentry-trace":"850e7def522d4a929af437d9b7e86294-464150540a0e44f7","baggage":"sentry-trace_id=850e7def522d4a929af437d9b7e86294,sentry-release=Unknown"},"correlation_id":"01M4JPTMMTR7GF3X2GQTJFQR1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.453Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2701ed23e7bba3716a49f037: done: 0.016885 sec","job_status":"done","queue_duration_s":0.414394,"scheduling_latency_s":0.414407,"redis_calls":7,"redis_duration_s":0.001647,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016885,"completed_at":"2026-10-10T10:48:52.885Z","load_balancing_strategy":"primary","db_duration_s":0.004085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1958gitlab # [ 256.128046] gitaly[892]: time="2026-10-10T10:48:53.283Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:23.281" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:53.282" grpc.time_ms=0.976 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1959gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc'1960gitlab # [ 256.146701] gitaly[892]: time="2026-10-10T10:48:53.302Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-10T10:49:23.294" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:53.294" grpc.time_ms=7.557 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 256.171126] gitaly[892]: time="2026-10-10T10:48:53.326Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:03.305" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:53.315" grpc.time_ms=11.282 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 256.197213] gitaly[892]: time="2026-10-10T10:48:53.352Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:49:03.012" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:53.352" grpc.time_ms=0.225 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 256.202664] nixbot[2666]: {"ts": "2026-10-10T10:48:53.357Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:57032 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1\" 200"}1964gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=49594e03ed656717508a1fe4aec0d95db48c51fc', in 0.06 seconds)1965(finished: subtest: push delivers the webhook, in 50.34 seconds)1966subtest: eval and build post commit statuses1967gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses1968gitlab # [ 256.213503] gitaly[892]: time="2026-10-10T10:48:53.369Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:49:03.015" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:53.356" grpc.time_ms=12.799 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969gitlab # [ 256.250398] gitaly[892]: time="2026-10-10T10:48:53.406Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:53.405" grpc.time_ms=0.395 pid=892 span.kind=server system=grpc1970gitlab # [ 256.278818] gitaly[892]: time="2026-10-10T10:48:53.434Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:53.434" grpc.time_ms=0.396 pid=892 span.kind=server system=grpc1971gitlab # [ 256.292444] gitaly[892]: time="2026-10-10T10:48:53.448Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW4WTK5SFGP4G0PQJTBW3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:23.047" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:53.447" grpc.time_ms=0.569 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 256.297226] gitaly[892]: time="2026-10-10T10:48:53.452Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 diskcache=7b285000-2033-47ce-adea-869b99557298 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:49:03.031" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:53.372" pid=892 ref=refs/keep-around/49594e03ed656717508a1fe4aec0d95db48c51fc remote_ip="2001:db8:1::1" revision=49594e03ed656717508a1fe4aec0d95db48c51fc span.kind=server system=grpc username=root1973gitlab # [ 256.297862] gitaly[892]: time="2026-10-10T10:48:53.453Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5207 command.oublock=8 command.real_time_ms=71 command.spawn_token_fork_ms=1 command.system_time_ms=29 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:49:03.031" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:53.372" grpc.time_ms=81.011 pid=892 ref=refs/keep-around/49594e03ed656717508a1fe4aec0d95db48c51fc remote_ip="2001:db8:1::1" revision=49594e03ed656717508a1fe4aec0d95db48c51fc span.kind=server system=grpc username=root1974gitlab # [ 256.303028] gitaly[892]: time="2026-10-10T10:48:53.458Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:23.458" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:53.458" grpc.time_ms=0.282 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 256.312410] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.466Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"978bba63a1a1a5909f0e1884","created_at":"2026-10-10T10:48:53.460Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-4c3b3a4ef3fa405b","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.463Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-978bba63a1a1a5909f0e1884: start","job_status":"start","queue_duration_s":0.002333,"scheduling_latency_s":0.002376}1976gitlab # [ 256.340552] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.496Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"978bba63a1a1a5909f0e1884","created_at":"2026-10-10T10:48:53.460Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-4c3b3a4ef3fa405b","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.463Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-978bba63a1a1a5909f0e1884: done: 0.030237 sec","job_status":"done","queue_duration_s":0.002333,"scheduling_latency_s":0.002376,"redis_calls":7,"redis_duration_s":0.001716,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000672,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030237,"completed_at":"2026-10-10T10:48:53.496Z","load_balancing_strategy":"primary","db_duration_s":0.005173,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1977gitlab # [ 256.351866] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.497Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"53fa10e37b864a18d654aa24","created_at":"2026-10-10T10:48:53.464Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-4c3b3a4ef3fa405b","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.475Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-53fa10e37b864a18d654aa24: start","job_status":"start","queue_duration_s":0.021788,"scheduling_latency_s":0.021805}1978gitlab # [ 256.401114] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 10661979gitlab # [ 256.413732] nixbot[2666]: {"ts": "2026-10-10T10:48:53.569Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-49594e03ed65-5"}1980gitlab # [ 256.432048] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1841981gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.23 seconds)1982gitlab # [ 256.480806] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.636Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"53fa10e37b864a18d654aa24","created_at":"2026-10-10T10:48:53.464Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-4c3b3a4ef3fa405b","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.475Z","job_size_bytes":65,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-53fa10e37b864a18d654aa24: done: 0.139141 sec","job_status":"done","queue_duration_s":0.021788,"scheduling_latency_s":0.021805,"redis_calls":10,"redis_duration_s":0.004561,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002451,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.00211,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.139141,"completed_at":"2026-10-10T10:48:53.636Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1983gitlab # [ 256.509990] nixbot[2666]: {"ts": "2026-10-10T10:48:53.665Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-49594e03ed65-5", "argv0": "bwrap"}1984gitlab # [ 256.523344] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.678Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ee8efaa1e4a93c6f6d73a6cd","created_at":"2026-10-10T10:48:53.540Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-4c3b3a4ef3fa405b","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.542Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ee8efaa1e4a93c6f6d73a6cd: start","job_status":"start","queue_duration_s":0.136257,"scheduling_latency_s":0.13629}1985gitlab # [ 256.587744] nix-daemon[2824]: accepted connection from pid 3651, user nixbot1986gitlab # [ 256.602729] gitaly[892]: time="2026-10-10T10:48:53.758Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:23.656" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:53.757" grpc.time_ms=1.059 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 256.635599] gitaly[892]: time="2026-10-10T10:48:53.791Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW409C3FRJH86F6F183R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:23.689" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:53.790" grpc.time_ms=0.827 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab # [ 256.657505] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.812Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ee8efaa1e4a93c6f6d73a6cd","created_at":"2026-10-10T10:48:53.540Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-4c3b3a4ef3fa405b","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.542Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ee8efaa1e4a93c6f6d73a6cd: done: 0.134145 sec","job_status":"done","queue_duration_s":0.136257,"scheduling_latency_s":0.13629,"gitaly_calls":2,"gitaly_duration_s":0.008269,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.016321,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000741,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000899,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000281,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000258,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.011531,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070764,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.094492736,"duration_s":0.134145,"completed_at":"2026-10-10T10:48:53.812Z","load_balancing_strategy":"primary","db_duration_s":0.028853,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1989gitlab # [ 256.706680] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"afb96f99dd1c012edf0d9482","created_at":"2026-10-10T10:48:53.740Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-84b4f0b2fa354af7","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.742Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-afb96f99dd1c012edf0d9482: start","job_status":"start","queue_duration_s":0.11812,"scheduling_latency_s":0.118155}1990gitlab # [ 256.770752] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:53.925Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"afb96f99dd1c012edf0d9482","created_at":"2026-10-10T10:48:53.740Z","trace_propagation_headers":{"sentry-trace":"6694e533c94d4c12adc0919f8f37c5f3-84b4f0b2fa354af7","baggage":"sentry-trace_id=6694e533c94d4c12adc0919f8f37c5f3,sentry-release=Unknown"},"correlation_id":"01M4JPW409C3FRJH86F6F183R6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.742Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-afb96f99dd1c012edf0d9482: done: 0.06559 sec","job_status":"done","queue_duration_s":0.11812,"scheduling_latency_s":0.118155,"redis_calls":7,"redis_duration_s":0.001745,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06559,"completed_at":"2026-10-10T10:48:53.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses1992gitlab # [ 257.551619] gitaly[892]: time="2026-10-10T10:48:54.706Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW63ADW8WZH38KDG7DMH1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:24.704" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:54.705" grpc.time_ms=1.091 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993gitlab # [ 257.588287] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1081994gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.16 seconds)1995gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses1996gitlab # [ 258.694108] gitaly[892]: time="2026-10-10T10:48:55.849Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW77B4NWVA238KGGRW8AN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:25.347" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:55.848" grpc.time_ms=0.881 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997gitlab # [ 258.755786] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1221998gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.17 seconds)1999gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses2000gitlab # [ 259.861974] gitaly[892]: time="2026-10-10T10:48:57.016Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW8BV9KD2JVBXCT45G4XQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:27.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:57.015" grpc.time_ms=0.985 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001gitlab # [ 259.888273] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 872002gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.13 seconds)2003gitlab # [ 260.224829] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"33024ff1277e813747c832ff","created_at":"2026-10-10T10:48:57.360Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f135e9b7963977d26f206f2083d1009","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.375Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-33024ff1277e813747c832ff: start","job_status":"start","queue_duration_s":0.00284,"scheduling_latency_s":0.002884}2004gitlab # [ 260.449985] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"33024ff1277e813747c832ff","created_at":"2026-10-10T10:48:57.360Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f135e9b7963977d26f206f2083d1009","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.375Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-33024ff1277e813747c832ff: done: 0.226754 sec","job_status":"done","queue_duration_s":0.00284,"scheduling_latency_s":0.002884,"redis_calls":13,"redis_duration_s":0.02522,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.012966,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007917,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004337,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001896331,"exclusive_lock_hold_duration_s":0.115739571,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.226754,"completed_at":"2026-10-10T10:48:57.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.07558,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2005gitlab # [ 260.530483] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.684Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"93ff37f89d1a964b70824067","created_at":"2026-10-10T10:48:57.420Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8a9f733303aba005db7e16ed2e63b6d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.440Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-93ff37f89d1a964b70824067: start","job_status":"start","queue_duration_s":0.244488,"scheduling_latency_s":0.244535}2006gitlab # [ 260.564114] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.719Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2007gitlab # [ 260.578467] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.733Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"93ff37f89d1a964b70824067","created_at":"2026-10-10T10:48:57.420Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8a9f733303aba005db7e16ed2e63b6d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.440Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-93ff37f89d1a964b70824067: done: 0.048687 sec","job_status":"done","queue_duration_s":0.244488,"scheduling_latency_s":0.244535,"redis_calls":9,"redis_duration_s":0.006941,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000724,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048687,"completed_at":"2026-10-10T10:48:57.733Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 260.622220] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.777Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d4d23b47a283269f5a391548","created_at":"2026-10-10T10:48:57.463Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be7fae3d924d2bc154934d12a59057ef","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.475Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d4d23b47a283269f5a391548: start","job_status":"start","queue_duration_s":0.301726,"scheduling_latency_s":0.301775}2009gitlab # [ 260.701842] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.856Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d4d23b47a283269f5a391548","created_at":"2026-10-10T10:48:57.463Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be7fae3d924d2bc154934d12a59057ef","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.475Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d4d23b47a283269f5a391548: done: 0.079461 sec","job_status":"done","queue_duration_s":0.301726,"scheduling_latency_s":0.301775,"redis_calls":12,"redis_duration_s":0.020949,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.0014,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003949,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003087543,"duration_s":0.079461,"completed_at":"2026-10-10T10:48:57.856Z","load_balancing_strategy":"primary","db_duration_s":0.00408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 260.767281] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f436f7ae3f6fdf7601cc6602","created_at":"2026-10-10T10:48:57.540Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cd5f677d8f63911debf069c68838f11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.552Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f436f7ae3f6fdf7601cc6602: start","job_status":"start","queue_duration_s":0.37052,"scheduling_latency_s":0.370556}2011gitlab # [ 260.822168] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:57.977Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2012gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses2013gitlab # [ 260.862937] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f436f7ae3f6fdf7601cc6602","created_at":"2026-10-10T10:48:57.540Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cd5f677d8f63911debf069c68838f11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.552Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f436f7ae3f6fdf7601cc6602: done: 0.095433 sec","job_status":"done","queue_duration_s":0.37052,"scheduling_latency_s":0.370556,"redis_calls":9,"redis_duration_s":0.025417,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003069,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018656,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003692,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.095433,"completed_at":"2026-10-10T10:48:58.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001297,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 260.931285] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"60bd64e69d2c40d73fdcfa72","created_at":"2026-10-10T10:48:57.561Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff7ecd0c231a24c5501d1ab27ffa7443","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.565Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-60bd64e69d2c40d73fdcfa72: start","job_status":"start","queue_duration_s":0.520758,"scheduling_latency_s":0.520814}2015gitlab # [ 260.995822] gitaly[892]: time="2026-10-10T10:48:58.150Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW9FBXB0SBKQN10XDBB5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:28.148" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:58.149" grpc.time_ms=0.97 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2016gitlab # [ 261.019928] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.175Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2017gitlab # [ 261.025942] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 892018gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.13 seconds)2019gitlab # [ 261.044090] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"60bd64e69d2c40d73fdcfa72","created_at":"2026-10-10T10:48:57.561Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff7ecd0c231a24c5501d1ab27ffa7443","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.565Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-60bd64e69d2c40d73fdcfa72: done: 0.112447 sec","job_status":"done","queue_duration_s":0.520758,"scheduling_latency_s":0.520814,"redis_calls":9,"redis_duration_s":0.007573,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00173,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002768,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003075,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.112447,"completed_at":"2026-10-10T10:48:58.198Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011397,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 261.071759] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"de083d2818239283623b6c14","created_at":"2026-10-10T10:48:57.571Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ecabdb376fbeb1eee0fed17374aa9a64","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.580Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-de083d2818239283623b6c14: start","job_status":"start","queue_duration_s":0.623766,"scheduling_latency_s":0.623807}2021gitlab # [ 261.126407] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.281Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"de083d2818239283623b6c14","created_at":"2026-10-10T10:48:57.571Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ecabdb376fbeb1eee0fed17374aa9a64","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.580Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-de083d2818239283623b6c14: done: 0.077436 sec","job_status":"done","queue_duration_s":0.623766,"scheduling_latency_s":0.623807,"redis_calls":9,"redis_duration_s":0.012204,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001494,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.077436,"completed_at":"2026-10-10T10:48:58.281Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022gitlab # [ 261.170621] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8dc7d410381b0c16af8ea147","created_at":"2026-10-10T10:48:57.590Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbf0eb924f70f9d6e2de1e599371c7d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.602Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8dc7d410381b0c16af8ea147: start","job_status":"start","queue_duration_s":0.722766,"scheduling_latency_s":0.722803}2023gitlab # [ 261.261210] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8dc7d410381b0c16af8ea147","created_at":"2026-10-10T10:48:57.590Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbf0eb924f70f9d6e2de1e599371c7d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.602Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8dc7d410381b0c16af8ea147: done: 0.090175 sec","job_status":"done","queue_duration_s":0.722766,"scheduling_latency_s":0.722803,"redis_calls":11,"redis_duration_s":0.014204999999999999,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004245,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002139,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090175,"completed_at":"2026-10-10T10:48:58.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 261.302480] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"869e2804acded4f1fd9e62fb","created_at":"2026-10-10T10:48:57.686Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d7e78c9b92fe1644ccf33a49b60bffb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.799Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-869e2804acded4f1fd9e62fb: start","job_status":"start","queue_duration_s":0.619773,"scheduling_latency_s":0.619817}2025gitlab # [ 261.348158] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"869e2804acded4f1fd9e62fb","created_at":"2026-10-10T10:48:57.686Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d7e78c9b92fe1644ccf33a49b60bffb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.799Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-869e2804acded4f1fd9e62fb: done: 0.084312 sec","job_status":"done","queue_duration_s":0.619773,"scheduling_latency_s":0.619817,"redis_calls":12,"redis_duration_s":0.048359,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.038216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002901765,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.084312,"completed_at":"2026-10-10T10:48:58.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 261.393755] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"86c940980bca499ac26ba8ee","created_at":"2026-10-10T10:48:57.817Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40b33ce984678e061d11bf5029543a53","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.833Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-86c940980bca499ac26ba8ee: start","job_status":"start","queue_duration_s":0.715598,"scheduling_latency_s":0.715647}2027gitlab # [ 261.496196] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"86c940980bca499ac26ba8ee","created_at":"2026-10-10T10:48:57.817Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40b33ce984678e061d11bf5029543a53","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.833Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-86c940980bca499ac26ba8ee: done: 0.102645 sec","job_status":"done","queue_duration_s":0.715598,"scheduling_latency_s":0.715647,"redis_calls":9,"redis_duration_s":0.011788,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001518,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003751,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.102645,"completed_at":"2026-10-10T10:48:58.651Z","load_balancing_strategy":"primary","db_duration_s":0.039132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 261.502244] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.657Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"626d69a81ff68bba7cb652eb","created_at":"2026-10-10T10:48:57.844Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.921Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-626d69a81ff68bba7cb652eb: start","job_status":"start","queue_duration_s":0.735949,"scheduling_latency_s":0.735987}2029gitlab # [ 261.539616] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"626d69a81ff68bba7cb652eb","created_at":"2026-10-10T10:48:57.844Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.921Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-626d69a81ff68bba7cb652eb: done: 0.037563 sec","job_status":"done","queue_duration_s":0.735949,"scheduling_latency_s":0.735987,"redis_calls":18,"redis_duration_s":0.0031,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000774,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000219,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001557,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037563,"completed_at":"2026-10-10T10:48:58.695Z","load_balancing_strategy":"primary","db_duration_s":0.003351,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 261.590547] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7709bdf471b78f3a05456b9f","created_at":"2026-10-10T10:48:57.950Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"692319cacb39217f46ff297f6cc3c701","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.969Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7709bdf471b78f3a05456b9f: start","job_status":"start","queue_duration_s":0.726689,"scheduling_latency_s":0.726708}2031gitlab # [ 261.591550] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7709bdf471b78f3a05456b9f","created_at":"2026-10-10T10:48:57.950Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"692319cacb39217f46ff297f6cc3c701","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:57.969Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7709bdf471b78f3a05456b9f: done: 0.013499 sec","job_status":"done","queue_duration_s":0.726689,"scheduling_latency_s":0.726708,"redis_calls":9,"redis_duration_s":0.001306,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013499,"completed_at":"2026-10-10T10:48:58.709Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2032gitlab # [ 261.644132] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a9ca37c1d25c53649aaa104a","created_at":"2026-10-10T10:48:58.001Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1eb21b790a511976c9e4ef175b9d79c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.020Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a9ca37c1d25c53649aaa104a: start","job_status":"start","queue_duration_s":0.690127,"scheduling_latency_s":0.690142}2033gitlab # [ 261.645242] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"a9ca37c1d25c53649aaa104a","created_at":"2026-10-10T10:48:58.001Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1eb21b790a511976c9e4ef175b9d79c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.020Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-a9ca37c1d25c53649aaa104a: done: 0.016558 sec","job_status":"done","queue_duration_s":0.690127,"scheduling_latency_s":0.690142,"redis_calls":9,"redis_duration_s":0.001327,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000262,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000295,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016558,"completed_at":"2026-10-10T10:48:58.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2034gitlab # [ 261.686638] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"da056828343fd90ae6afbeda","created_at":"2026-10-10T10:48:58.129Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eb659a1bc787aba418549335d02412d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.142Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-da056828343fd90ae6afbeda: start","job_status":"start","queue_duration_s":0.58603,"scheduling_latency_s":0.586046}2035gitlab # [ 261.687666] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.745Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"da056828343fd90ae6afbeda","created_at":"2026-10-10T10:48:58.129Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eb659a1bc787aba418549335d02412d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.142Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-da056828343fd90ae6afbeda: done: 0.017337 sec","job_status":"done","queue_duration_s":0.58603,"scheduling_latency_s":0.586046,"redis_calls":11,"redis_duration_s":0.001596,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000320153,"exclusive_lock_hold_duration_s":0.004975492,"duration_s":0.017337,"completed_at":"2026-10-10T10:48:58.745Z","load_balancing_strategy":"primary","db_duration_s":0.003169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2036gitlab # [ 261.706239] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2ba1f62d59ab835db8dd114c","created_at":"2026-10-10T10:48:58.165Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d71dc1b0a2adb92a970f130295d9cb97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T10:48:58.183Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2ba1f62d59ab835db8dd114c: start","job_status":"start","queue_duration_s":0.56332,"scheduling_latency_s":0.563334}2037gitlab # [ 261.718601] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.756Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2038gitlab # [ 261.721743] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.760Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"2ba1f62d59ab835db8dd114c","created_at":"2026-10-10T10:48:58.165Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d71dc1b0a2adb92a970f130295d9cb97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T10:48:58.183Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-2ba1f62d59ab835db8dd114c: done: 0.014046 sec","job_status":"done","queue_duration_s":0.56332,"scheduling_latency_s":0.563334,"redis_calls":9,"redis_duration_s":0.001221,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014046,"completed_at":"2026-10-10T10:48:58.760Z","load_balancing_strategy":"primary","db_duration_s":0.002508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab # [ 261.764873] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6a10a03545f408347f7e08e2","created_at":"2026-10-10T10:48:58.195Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11fb794b87c1e3b813a070d532372c9a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.205Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6a10a03545f408347f7e08e2: start","job_status":"start","queue_duration_s":0.555562,"scheduling_latency_s":0.555576}2040gitlab # [ 261.771292] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6a10a03545f408347f7e08e2","created_at":"2026-10-10T10:48:58.195Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11fb794b87c1e3b813a070d532372c9a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.205Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6a10a03545f408347f7e08e2: done: 0.161216 sec","job_status":"done","queue_duration_s":0.555562,"scheduling_latency_s":0.555576,"redis_calls":13,"redis_duration_s":0.00212,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000355353,"exclusive_lock_hold_duration_s":0.144729847,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.161216,"completed_at":"2026-10-10T10:48:58.922Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.061184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2041gitlab # [ 261.839626] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8a32a077918aa25e75023564","created_at":"2026-10-10T10:48:58.219Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a112151808766c2151849c118ec167bd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:48:58.223Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8a32a077918aa25e75023564: start","job_status":"start","queue_duration_s":0.700569,"scheduling_latency_s":0.700586}2042gitlab # [ 261.839959] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8a32a077918aa25e75023564","created_at":"2026-10-10T10:48:58.219Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a112151808766c2151849c118ec167bd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:48:58.223Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8a32a077918aa25e75023564: done: 0.023939 sec","job_status":"done","queue_duration_s":0.700569,"scheduling_latency_s":0.700586,"redis_calls":9,"redis_duration_s":0.001452,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023939,"completed_at":"2026-10-10T10:48:58.947Z","load_balancing_strategy":"primary","db_duration_s":0.004936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043gitlab # [ 261.897852] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:58.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"5d04b4cc77728c785cf27795","created_at":"2026-10-10T10:48:58.241Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.251Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-5d04b4cc77728c785cf27795: start","job_status":"start","queue_duration_s":0.697424,"scheduling_latency_s":0.697442}2044gitlab # [ 261.923523] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"5d04b4cc77728c785cf27795","created_at":"2026-10-10T10:48:58.241Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.251Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-5d04b4cc77728c785cf27795: done: 0.130041 sec","job_status":"done","queue_duration_s":0.697424,"scheduling_latency_s":0.697442,"redis_calls":65,"redis_duration_s":0.011822,"redis_read_bytes":114,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000561,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000927,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009807,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000342781,"duration_s":0.130041,"completed_at":"2026-10-10T10:48:59.078Z","load_balancing_strategy":"primary","db_duration_s":0.001102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2045gitlab # [ 261.927663] nix-daemon[2824]: accepted connection from pid 3688, user nixbot2046gitlab # [ 261.939387] nix-daemon[2824]: accepted connection from pid 3651, user nixbot2047gitlab # [ 261.967088] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"276c40f5fdb177c9e1e722ed","created_at":"2026-10-10T10:48:58.256Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47b5c04cf0956d37268efa1c11f4f5ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.261Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-276c40f5fdb177c9e1e722ed: start","job_status":"start","queue_duration_s":0.859278,"scheduling_latency_s":0.859309}2048gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses2049gitlab # [ 262.057452] gitaly[892]: time="2026-10-10T10:48:59.212Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPWAGZJ04WH93KWJZPDW7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:29.211" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:59.212" grpc.time_ms=0.559 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2050gitlab # [ 262.072819] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.228Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"276c40f5fdb177c9e1e722ed","created_at":"2026-10-10T10:48:58.256Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47b5c04cf0956d37268efa1c11f4f5ae","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.261Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-276c40f5fdb177c9e1e722ed: done: 0.107633 sec","job_status":"done","queue_duration_s":0.859278,"scheduling_latency_s":0.859309,"redis_calls":70,"redis_duration_s":0.015258999999999998,"redis_read_bytes":119,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003008,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000454,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.0113,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.107633,"completed_at":"2026-10-10T10:48:59.228Z","load_balancing_strategy":"primary","db_duration_s":0.002119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051gitlab # [ 262.073870] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"5862afac3408cb6610b67c86","created_at":"2026-10-10T10:48:58.263Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1df6aee16398233e464ff0308fb14cfc","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.271Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-5862afac3408cb6610b67c86: start","job_status":"start","queue_duration_s":0.957963,"scheduling_latency_s":0.957983}2052gitlab # [ 262.093658] gitaly[892]: time="2026-10-10T10:48:59.249Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPWAGZJ04WH93KWJZPDW7S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:09.231" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:59.242" grpc.time_ms=6.891 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 262.133592] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"5862afac3408cb6610b67c86","created_at":"2026-10-10T10:48:58.263Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1df6aee16398233e464ff0308fb14cfc","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.271Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-5862afac3408cb6610b67c86: done: 0.019927 sec","job_status":"done","queue_duration_s":0.957963,"scheduling_latency_s":0.957983,"redis_calls":10,"redis_duration_s":0.0014620000000000002,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221257,"duration_s":0.019927,"completed_at":"2026-10-10T10:48:59.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 262.134737] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.250Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"98c5b333637abfbf20cf0b89","created_at":"2026-10-10T10:48:58.282Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db389f656bb6cd00721df56dd8b3d80f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.345Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-98c5b333637abfbf20cf0b89: start","job_status":"start","queue_duration_s":0.904865,"scheduling_latency_s":0.904878}2055gitlab # [ 262.182666] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.256Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2056gitlab # [ 262.185582] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1192057gitlab # [ 262.185923] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"98c5b333637abfbf20cf0b89","created_at":"2026-10-10T10:48:58.282Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db389f656bb6cd00721df56dd8b3d80f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.345Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-98c5b333637abfbf20cf0b89: done: 0.011095 sec","job_status":"done","queue_duration_s":0.904865,"scheduling_latency_s":0.904878,"redis_calls":9,"redis_duration_s":0.001784,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011095,"completed_at":"2026-10-10T10:48:59.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2058gitlab # [ 262.227615] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f68d8f0123dd5b3d0ba97ed5","created_at":"2026-10-10T10:48:58.371Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7078c9df6e159dd6a6f606182688b71b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.381Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f68d8f0123dd5b3d0ba97ed5: start","job_status":"start","queue_duration_s":0.880224,"scheduling_latency_s":0.880238}2059gitlab # [ 262.240397] gitaly[892]: time="2026-10-10T10:48:59.396Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPWAMZE8W982BS7K6B3618 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:29.394" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:59.395" grpc.time_ms=0.638 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060gitlab # [ 262.278815] gitaly[892]: time="2026-10-10T10:48:59.434Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPWAMZE8W982BS7K6B3618 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:09.087" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:59.427" grpc.time_ms=6.948 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061gitlab # [ 262.295597] gitaly[892]: time="2026-10-10T10:48:59.451Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPWAJKR4079FRY9D1RV3K8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:29.050" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:59.450" grpc.time_ms=0.508 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2062gitlab # [ 262.326500] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1862063gitlab # [ 262.336565] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 2722064gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.32 seconds)2065gitlab # [ 262.373564] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f68d8f0123dd5b3d0ba97ed5","created_at":"2026-10-10T10:48:58.371Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7078c9df6e159dd6a6f606182688b71b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.381Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f68d8f0123dd5b3d0ba97ed5: done: 0.266535 sec","job_status":"done","queue_duration_s":0.880224,"scheduling_latency_s":0.880238,"redis_calls":564,"redis_duration_s":0.08539699999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.084252,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.191199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.266535,"completed_at":"2026-10-10T10:48:59.528Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2066gitlab # [ 262.374699] gitaly[892]: time="2026-10-10T10:48:59.530Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPWAV8F04T1SBCBY5RDCDB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:29.529" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:59.529" grpc.time_ms=0.628 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 262.407606] gitaly[892]: time="2026-10-10T10:48:59.563Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPWAV8F04T1SBCBY5RDCDB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:09.217" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:59.557" grpc.time_ms=5.964 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068gitlab # [ 262.436200] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.591Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"6b970cbca5a6e13be39b6a04","created_at":"2026-10-10T10:48:58.398Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"f921b14994374f3ad8b43cc46ae5c131","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.413Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-6b970cbca5a6e13be39b6a04: start","job_status":"start","queue_duration_s":1.178063,"scheduling_latency_s":1.178105}2069gitlab # [ 262.449982] gitlab-workhorse[828]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/49594e03ed656717508a1fe4aec0d95db48c51fc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 992070gitlab # [ 262.455570] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.611Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"6b970cbca5a6e13be39b6a04","created_at":"2026-10-10T10:48:58.398Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"f921b14994374f3ad8b43cc46ae5c131","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.413Z","job_size_bytes":3,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-6b970cbca5a6e13be39b6a04: done: 0.019799 sec","job_status":"done","queue_duration_s":1.178063,"scheduling_latency_s":1.178105,"redis_calls":10,"redis_duration_s":0.001886,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000307023,"duration_s":0.019799,"completed_at":"2026-10-10T10:48:59.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 262.456777] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.612Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"91fcfe7eab6ccc2e8ad0b596","created_at":"2026-10-10T10:48:58.422Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"972562318e17dd39bc6958ba38abdf0a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:48:58.436Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-91fcfe7eab6ccc2e8ad0b596: start","job_status":"start","queue_duration_s":1.176112,"scheduling_latency_s":1.176126}2072gitlab # [ 262.465127] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.620Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2073gitlab # [ 262.503661] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"91fcfe7eab6ccc2e8ad0b596","created_at":"2026-10-10T10:48:58.422Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"972562318e17dd39bc6958ba38abdf0a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:48:58.436Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-91fcfe7eab6ccc2e8ad0b596: done: 0.012674 sec","job_status":"done","queue_duration_s":1.176112,"scheduling_latency_s":1.176126,"redis_calls":9,"redis_duration_s":0.00145,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012674,"completed_at":"2026-10-10T10:48:59.624Z","load_balancing_strategy":"primary","db_duration_s":0.000314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074gitlab # [ 262.504595] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ed885a8d812103cb521a67fe","created_at":"2026-10-10T10:48:58.451Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3adfc8319534438ce09862f811fc3a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.495Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ed885a8d812103cb521a67fe: start","job_status":"start","queue_duration_s":1.130732,"scheduling_latency_s":1.130748}2075gitlab # [ 262.548881] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ed885a8d812103cb521a67fe","created_at":"2026-10-10T10:48:58.451Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3adfc8319534438ce09862f811fc3a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.495Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ed885a8d812103cb521a67fe: done: 0.014631 sec","job_status":"done","queue_duration_s":1.130732,"scheduling_latency_s":1.130748,"redis_calls":11,"redis_duration_s":0.001617,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000682,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000320431,"exclusive_lock_hold_duration_s":0.002272076,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.014631,"completed_at":"2026-10-10T10:48:59.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 262.555098] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7705b920407bc667e1d087d3","created_at":"2026-10-10T10:48:58.548Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6315e86404c653b8bbc1b5b6a607190b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.578Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7705b920407bc667e1d087d3: start","job_status":"start","queue_duration_s":1.063094,"scheduling_latency_s":1.063108}2077gitlab # [ 262.601688] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7705b920407bc667e1d087d3","created_at":"2026-10-10T10:48:58.548Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6315e86404c653b8bbc1b5b6a607190b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.578Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7705b920407bc667e1d087d3: done: 0.02398 sec","job_status":"done","queue_duration_s":1.063094,"scheduling_latency_s":1.063108,"redis_calls":11,"redis_duration_s":0.001534,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000621,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0003696,"exclusive_lock_hold_duration_s":0.007666896,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.02398,"completed_at":"2026-10-10T10:48:59.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 262.606915] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9a86190b577fa865e1d3f19c","created_at":"2026-10-10T10:48:58.606Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1bd5ad1649e873a261d0d55528c92a2e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.621Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9a86190b577fa865e1d3f19c: start","job_status":"start","queue_duration_s":1.044612,"scheduling_latency_s":1.044626}2079gitlab # [ 262.654514] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9a86190b577fa865e1d3f19c","created_at":"2026-10-10T10:48:58.606Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1bd5ad1649e873a261d0d55528c92a2e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.621Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9a86190b577fa865e1d3f19c: done: 0.019018 sec","job_status":"done","queue_duration_s":1.044612,"scheduling_latency_s":1.044626,"redis_calls":12,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005563836,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019018,"completed_at":"2026-10-10T10:48:59.685Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001586,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2080gitlab # [ 262.659521] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"bdc6200c003e74baca0e96ca","created_at":"2026-10-10T10:48:58.631Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a309a85e1719f723871a3964b06e4a0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.648Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-bdc6200c003e74baca0e96ca: start","job_status":"start","queue_duration_s":1.03806,"scheduling_latency_s":1.038074}2081gitlab # [ 262.700291] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"bdc6200c003e74baca0e96ca","created_at":"2026-10-10T10:48:58.631Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-02a4be6e471e4de9","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a309a85e1719f723871a3964b06e4a0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.648Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-bdc6200c003e74baca0e96ca: done: 0.014952 sec","job_status":"done","queue_duration_s":1.03806,"scheduling_latency_s":1.038074,"redis_calls":12,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000563,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000343619,"exclusive_lock_hold_duration_s":0.00319482,"duration_s":0.014952,"completed_at":"2026-10-10T10:48:59.701Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082gitlab # [ 262.710670] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.702Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0dbd66a06f4af24bf05032db","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0dbd66a06f4af24bf05032db: start","job_status":"start","queue_duration_s":1.012491,"scheduling_latency_s":1.012505}2083gitlab # [ 262.758734] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.712Z","message":"No existing merge request to be cleaned up."}2084gitlab # [ 262.759542] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.720Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0dbd66a06f4af24bf05032db","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0dbd66a06f4af24bf05032db: done: 0.01808 sec","job_status":"done","queue_duration_s":1.012491,"scheduling_latency_s":1.012505,"redis_calls":12,"redis_duration_s":0.001757,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000844,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01808,"completed_at":"2026-10-10T10:48:59.720Z","load_balancing_strategy":"primary","db_duration_s":0.002035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 262.760660] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.721Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d10ac18f74c9fcb1811a2522","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d10ac18f74c9fcb1811a2522: start","job_status":"start","queue_duration_s":1.031495,"scheduling_latency_s":1.03151}2086gitlab # [ 262.769779] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.728Z","message":"No existing merge request to be cleaned up."}2087gitlab # [ 262.807917] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.733Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d10ac18f74c9fcb1811a2522","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d10ac18f74c9fcb1811a2522: done: 0.011665 sec","job_status":"done","queue_duration_s":1.031495,"scheduling_latency_s":1.03151,"redis_calls":10,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011665,"completed_at":"2026-10-10T10:48:59.733Z","load_balancing_strategy":"primary","db_duration_s":0.001094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088gitlab # [ 262.808967] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.733Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9093a02da03cc3e8a0e56f5e","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9093a02da03cc3e8a0e56f5e: start","job_status":"start","queue_duration_s":1.044081,"scheduling_latency_s":1.044095}2089gitlab # [ 262.816231] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.740Z","message":"No existing merge request to be cleaned up."}2090gitlab # [ 262.819724] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.745Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9093a02da03cc3e8a0e56f5e","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9093a02da03cc3e8a0e56f5e: done: 0.011072 sec","job_status":"done","queue_duration_s":1.044081,"scheduling_latency_s":1.044095,"redis_calls":10,"redis_duration_s":0.0013219999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000783,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011072,"completed_at":"2026-10-10T10:48:59.745Z","load_balancing_strategy":"primary","db_duration_s":0.001019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab # [ 262.861536] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.745Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6a66e2fe461f5ca62885c22a","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6a66e2fe461f5ca62885c22a: start","job_status":"start","queue_duration_s":1.056052,"scheduling_latency_s":1.056066}2092gitlab # [ 262.863674] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.752Z","message":"No existing merge request to be cleaned up."}2093gitlab # [ 262.865164] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.757Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6a66e2fe461f5ca62885c22a","created_at":"2026-10-10T10:48:58.677Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-273d72fdbcf94aeb","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5ee466aa0b9885aa6dd18f671a3e8ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:58.689Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6a66e2fe461f5ca62885c22a: done: 0.012021 sec","job_status":"done","queue_duration_s":1.056052,"scheduling_latency_s":1.056066,"redis_calls":10,"redis_duration_s":0.001442,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012021,"completed_at":"2026-10-10T10:48:59.757Z","load_balancing_strategy":"primary","db_duration_s":0.001878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2094gitlab # [ 262.934242] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"38f95faaf87b450b0e5ebd93","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-38f95faaf87b450b0e5ebd93: start","job_status":"start","queue_duration_s":0.688007,"scheduling_latency_s":0.688021}2095gitlab # [ 262.935137] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"38f95faaf87b450b0e5ebd93","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-38f95faaf87b450b0e5ebd93: done: 0.026171 sec","job_status":"done","queue_duration_s":0.688007,"scheduling_latency_s":0.688021,"redis_calls":13,"redis_duration_s":0.0030180000000000003,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000421,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001443,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000293613,"duration_s":0.026171,"completed_at":"2026-10-10T10:48:59.785Z","load_balancing_strategy":"primary","db_duration_s":0.005158,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 262.980300] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"01d17f43ec6fa50ab57c2162","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-01d17f43ec6fa50ab57c2162: start","job_status":"start","queue_duration_s":0.715634,"scheduling_latency_s":0.715651}2097gitlab # [ 262.981168] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"01d17f43ec6fa50ab57c2162","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-01d17f43ec6fa50ab57c2162: done: 0.013289 sec","job_status":"done","queue_duration_s":0.715634,"scheduling_latency_s":0.715651,"redis_calls":10,"redis_duration_s":0.0020559999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013289,"completed_at":"2026-10-10T10:48:59.800Z","load_balancing_strategy":"primary","db_duration_s":0.001427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098gitlab # [ 262.992156] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6da7d680efe1fdf2efdbde8b","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6da7d680efe1fdf2efdbde8b: start","job_status":"start","queue_duration_s":0.730076,"scheduling_latency_s":0.73009}2099gitlab # [ 263.003530] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6da7d680efe1fdf2efdbde8b","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6da7d680efe1fdf2efdbde8b: done: 0.014065 sec","job_status":"done","queue_duration_s":0.730076,"scheduling_latency_s":0.73009,"redis_calls":10,"redis_duration_s":0.00218,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014065,"completed_at":"2026-10-10T10:48:59.815Z","load_balancing_strategy":"primary","db_duration_s":0.001061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100gitlab # [ 263.063981] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3a635cb6f545562add0237d7","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3a635cb6f545562add0237d7: start","job_status":"start","queue_duration_s":0.745328,"scheduling_latency_s":0.745351}2101gitlab # [ 263.064688] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3a635cb6f545562add0237d7","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3a635cb6f545562add0237d7: done: 0.014347 sec","job_status":"done","queue_duration_s":0.745328,"scheduling_latency_s":0.745351,"redis_calls":10,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014347,"completed_at":"2026-10-10T10:48:59.830Z","load_balancing_strategy":"primary","db_duration_s":0.002275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 263.086256] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e21855153b2e3debe0993f1d","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e21855153b2e3debe0993f1d: start","job_status":"start","queue_duration_s":0.760579,"scheduling_latency_s":0.760593}2103gitlab # [ 263.135848] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e21855153b2e3debe0993f1d","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e21855153b2e3debe0993f1d: done: 0.012246 sec","job_status":"done","queue_duration_s":0.760579,"scheduling_latency_s":0.760593,"redis_calls":10,"redis_duration_s":0.0013679999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012246,"completed_at":"2026-10-10T10:48:59.844Z","load_balancing_strategy":"primary","db_duration_s":0.001661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104gitlab # [ 263.136995] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"012760318c4815a2e631b8c7","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-012760318c4815a2e631b8c7: start","job_status":"start","queue_duration_s":0.773711,"scheduling_latency_s":0.773726}2105gitlab # [ 263.143662] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"012760318c4815a2e631b8c7","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-012760318c4815a2e631b8c7: done: 0.01208 sec","job_status":"done","queue_duration_s":0.773711,"scheduling_latency_s":0.773726,"redis_calls":10,"redis_duration_s":0.0013930000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01208,"completed_at":"2026-10-10T10:48:59.857Z","load_balancing_strategy":"primary","db_duration_s":0.001923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 263.210111] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9739a617e38e74f35fda948f","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9739a617e38e74f35fda948f: start","job_status":"start","queue_duration_s":0.786668,"scheduling_latency_s":0.786683}2107gitlab # [ 263.210765] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9739a617e38e74f35fda948f","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9739a617e38e74f35fda948f: done: 0.013601 sec","job_status":"done","queue_duration_s":0.786668,"scheduling_latency_s":0.786683,"redis_calls":10,"redis_duration_s":0.001469,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013601,"completed_at":"2026-10-10T10:48:59.871Z","load_balancing_strategy":"primary","db_duration_s":0.003094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108gitlab # [ 263.266850] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e2fd79aa98a4cb31de6447c6","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e2fd79aa98a4cb31de6447c6: start","job_status":"start","queue_duration_s":0.801192,"scheduling_latency_s":0.801207}2109gitlab # [ 263.267690] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e2fd79aa98a4cb31de6447c6","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e2fd79aa98a4cb31de6447c6: done: 0.01384 sec","job_status":"done","queue_duration_s":0.801192,"scheduling_latency_s":0.801207,"redis_calls":10,"redis_duration_s":0.002554,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01384,"completed_at":"2026-10-10T10:48:59.886Z","load_balancing_strategy":"primary","db_duration_s":0.001719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 263.281653] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8a8e0021cf7a3619572114d3","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8a8e0021cf7a3619572114d3: start","job_status":"start","queue_duration_s":0.81595,"scheduling_latency_s":0.815964}2111gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses2112gitlab # [ 263.322334] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8a8e0021cf7a3619572114d3","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8a8e0021cf7a3619572114d3: done: 0.02486 sec","job_status":"done","queue_duration_s":0.81595,"scheduling_latency_s":0.815964,"redis_calls":10,"redis_duration_s":0.0022630000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001458,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02486,"completed_at":"2026-10-10T10:48:59.912Z","load_balancing_strategy":"primary","db_duration_s":0.002619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113gitlab # [ 263.326683] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"51c03ce95fd4f4c50bec739a","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-51c03ce95fd4f4c50bec739a: start","job_status":"start","queue_duration_s":0.843154,"scheduling_latency_s":0.84319}2114gitlab # [ 263.367603] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"51c03ce95fd4f4c50bec739a","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-51c03ce95fd4f4c50bec739a: done: 0.017959 sec","job_status":"done","queue_duration_s":0.843154,"scheduling_latency_s":0.84319,"redis_calls":10,"redis_duration_s":0.002047,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017959,"completed_at":"2026-10-10T10:48:59.932Z","load_balancing_strategy":"primary","db_duration_s":0.002517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115gitlab # [ 263.369694] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b8b3ecbafd8b6bb2711dcb2","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b8b3ecbafd8b6bb2711dcb2: start","job_status":"start","queue_duration_s":0.862802,"scheduling_latency_s":0.862842}2116gitlab # [ 263.381198] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b8b3ecbafd8b6bb2711dcb2","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b8b3ecbafd8b6bb2711dcb2: done: 0.023874 sec","job_status":"done","queue_duration_s":0.862802,"scheduling_latency_s":0.862842,"redis_calls":10,"redis_duration_s":0.001905,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023874,"completed_at":"2026-10-10T10:48:59.957Z","load_balancing_strategy":"primary","db_duration_s":0.001818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2117gitlab # [ 263.415758] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"70ab60a12a018cf27762f1b0","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-70ab60a12a018cf27762f1b0: start","job_status":"start","queue_duration_s":0.888498,"scheduling_latency_s":0.888525}2118gitlab # [ 263.457124] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"70ab60a12a018cf27762f1b0","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-70ab60a12a018cf27762f1b0: done: 0.01497 sec","job_status":"done","queue_duration_s":0.888498,"scheduling_latency_s":0.888525,"redis_calls":10,"redis_duration_s":0.001657,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01497,"completed_at":"2026-10-10T10:48:59.974Z","load_balancing_strategy":"primary","db_duration_s":0.001423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab # [ 263.460480] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0a32930c14747fc62f5c00f8","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0a32930c14747fc62f5c00f8: start","job_status":"start","queue_duration_s":0.905001,"scheduling_latency_s":0.905023}2120gitlab # [ 263.471839] gitaly[892]: time="2026-10-10T10:49:00.594Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPWBWDRGFE64N87EC557WK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:30.592" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:49:00.593" grpc.time_ms=0.554 pid=892 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2121gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/49594e03ed656717508a1fe4aec0d95db48c51fc/statuses, in 0.19 seconds)2122(finished: subtest: eval and build post commit statuses, in 7.32 seconds)2123(finished: run the VM test script, in 339.79 seconds)2124gitlab # [ 263.482913] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0a32930c14747fc62f5c00f8","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0a32930c14747fc62f5c00f8: done: 0.013658 sec","job_status":"done","queue_duration_s":0.905001,"scheduling_latency_s":0.905023,"redis_calls":10,"redis_duration_s":0.001579,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013658,"completed_at":"2026-10-10T10:48:59.989Z","load_balancing_strategy":"primary","db_duration_s":0.001362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 263.524645] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:48:59.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"49956e90c2be8cda3700ddde","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-49956e90c2be8cda3700ddde: start","job_status":"start","queue_duration_s":0.919751,"scheduling_latency_s":0.919765}2126test script finished in 339.84s2127cleanup2128kill QemuMachine (pid 46)2129gitlab # [ 263.570417] sidekiq-cluster[2540]: {"severity":"INFO","time":"2026-10-10T10:49:00.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"49956e90c2be8cda3700ddde","created_at":"2026-10-10T10:48:58.957Z","trace_propagation_headers":{"sentry-trace":"c7d4cdc1eb7f49db92a08db0aa20c515-59a68fded23546d4","baggage":"sentry-trace_id=c7d4cdc1eb7f49db92a08db0aa20c515,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"2b71e3c4f2af11d51f1ebbc5145e5376","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:59.071Z","job_size_bytes":2,"pid":2540,"sidekiq_tid":"m7y0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-49956e90c2be8cda3700ddde: done: 0.011377 sec","job_status":"done","queue_duration_s":0.919751,"scheduling_latency_s":0.919765,"redis_calls":10,"redis_duration_s":0.00148,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2130(finished: cleanup, in 1.69 seconds)2131additionally exposed symbols:2132 gitlab,2133 vlan1,2134 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2135[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-10T10:48:02.602Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2136[]2137[]2138[]2139[]2140[]2141[]2142[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-10T10:48:52.483096Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2143[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2144[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2145[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2146[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2147[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2148[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': '2026-10-10T10:48:59.263Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-10-10T10:48:59.462Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2149[{'id': 5, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-10T10:48:53.533Z', 'started_at': None, 'finished_at': '2026-10-10T10:48:59.263Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '49594e03ed656717508a1fe4aec0d95db48c51fc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-10T10:48:59.462Z', 'started_at': None, 'finished_at': '2026-10-10T10:48:59.578Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]